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ABSTRACT 

This monograph contains four position papers and two 
conference papers from the Systemic Evaluation Project (fiscal year, 
1984) on the methodological issues associated with applying a 
comprehensive information system in a secondary school setting. These 
papers represent work in progress at a suburban Los Angeles high 
school field test of systemic evaluation methods. The feasibility 
study is being carried out by the Center for N the Study of Evaluation 
with the Laboratory in School and Community Relations. The four 
position papers are: (1) "Contextual Influences in Developing a 
School-Based Comprehensive Information System," by Donald H. 
Dorr-Bremme; (2) "The Use of Survey Data in Comprehensive Information 
Systems for Local School Improvement: Purposes r Practices r and 
Problems," by Leigh Burstein and Kenneth A. Sirotnik; (3) "Making 
Sense Out of Comprehensive School-Based Information Systems: 
Exploring Analyses and Reporting Methods for School Staff," by 
Sirotnik and Burstein; and (4) "Distributive Processing Issues in 
Education Information Systems," by Philip B. Ender. The conference 
papers are: (1) "Information Use in Local School Improvement: A 
Multilevel Perspective," by Burstein; and (2) "Using vs. Being Used 
by School Information Systems," by Sirotnik. (BS) 
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INTRODUCTION 

The six papers 1n this monograph constitute the four proposed 
deliverables for FY 1984 of the Systemic Evaluation Project plus the 
two papers delivered at the 1984 conference of the American 
Educational Research Association 1n the symposium "Comprehensive 
Information Systems for Districts and Schools: Ingredients, Uses, and 
Misuses" organized by Ken Sirotnik and Leigh Bursteln as part of this 
project* 

As stated 1n the CSE Institutional Prospectus 1984-1985 (February 
29, 1984, p. 10), a series of position papers was planned on 
methodological issues associated with applying a comprehensive 
information system in a secondary school setting. The specific issues 
to be addressed were as follows: 

1. School's organizational context surrounding the initiation of 
improvement efforts. 

2. The collection of survey data to augment the existing system 
and the problems that arise in integrating it into the system. 

3. Data analysis and report formatting enhancements that make the 
system more accessible and meaningful. 

4. The results of a technology audit of the hardware, software, 
and human resource issues as part of the activities of 
examining issues involved in making the information system an 
on-line, user-friendly adjunct to decision-making at the 
school level given the exigencies (equipment, personnel, 
perceptions, priorities, costs) of data processing 
capabilities in the district and the school. 

These issues constitute the foci, respectively, of the first four 

papers in this project report. 



It should be noted that although one or two authors (as listed) 
took primary responsibility for writing each report, all CSE staff on 
this project have worked together and contributed to all phases of the 
effort. Furthermore, these papers represent work in progress and, as 
such, must be considered preliminary in light of the data and 
ex^rience to be gained during the coming year. 

Nonetheless, the work represented in these reports constitutes 
substantial progress in coming to grips with practical concerns that 
arise when working withm the realities of schooling. This work, 
therefore, when modified and augmented by the data and experiences 
derived during FY 1985, will constitute the core of the FY 1985 
deliverables. In addition, the clinical-social perspective on 
informatioi selection and use a theme that pervades these reports 
— will provide a fifth focus for FY 1985 deliverables. 
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Contextual Influences in 
Developing a School -Bassd Comprehensive Information System 

Donald W. Dorr-Bremme 
INTRODUCTION 

The purpose of this paper is to identify and examine some social 
contextual factors that impinge upon a school's development of 
on-going, systemic evaluation for instructional decision making. The 
data for this analysis come from the case study of a "reality test" of 
such a system now under way in a suburban, California high school. 

The paper begins with a brief description of the school district 
and school in which this effort is being conducted, a succinct review 
of the model of systemic evaluation being tested there, and an outline 
of the reality test itself. It then describes and documents several 
social phenomena that contextual ize this effort and goes on to suggest 
how each appears to influence the school's response to systemic 
evaluation. 

All this should be construed as a progress report, rather than a 
list of findings. Tne reality test {or feasibility study) has been in 
progress one year and has another year yet to run. Nevertheless, both 
experience and related research suggest that the issues identified 
thus far in the project are important ones and that they tend to arise 
in many school districts which are engaged in efforts to develop 
instructional information systems. 

The Setting 

The reality test of systemic evaluation is being conducted at 
Site A in the Valley Unified District* on the fringe of the Los 



* A pseudonym 
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Angeles metropolitan area. The District serves an enrollment of about 
25,000 students, who come from a socioeconomically diverse community. 

Site A is one of two hign schools in the District. Among its 
2,000 or so students, approximately 85 percent are "Anglo"; the rest 
are Hispanic, Asian, and Black. School authorities estimate that 8 
to 10 percent of Site A's graduating seniors go on to attend four-year 
colleges or universities, another 20 to 30 percent attend a local 
community coll ege. 

The school prides itself on offering a full range of advanced 
placement courses, a rich curriculum, successful athletic programs, 
and a wide range of other extracurricular activities. Educators at 
Site A and in the District, however, express some concern that Site A 
graduates may not be pursuing post-secondary educational and career 
opportunities concomitant with the quality of school programs. 

While in most ways neither Site A or the Valley Districtare 
unusual, some special resources and programs make them propitious 
settings for a trial of the systemic evaluation concept. 

One distinctive feature of the resources at Site A is the 
presence of a computerized individual information system. CASA 
(Computerized Accountability for Student Achievement) is a 
computer-based student information management system which provides 
counseling staff and assistant principals instantaneous access to a 
wealth of information stored on each student. Through this system, 
counseling staff and adminstrators can access such student-level 
information as standardized test scores for multiple years {including 
information prior to entry to high school); proficiency testing 
information (from the District's Computer Managed Instruction program, 

8 
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which monitors student progress in Grades K-10 through the use of 
CRTs based on District continuum); curriculum and performance 
information including courses taken, credits, grades and class rank; 
background information including parental occupations, family size, 
census tract location of residence, and ethnicity; current school 
status information including eligibility for special programs (Gifted 
and Talented, Bilingual, Special Education) special school activities 
(athletics, school paper, etc), complete attendance history and 
referrals to various school services (psychologist, health office 
counselor and guidance office, principal, etc.). 

The CASA system was developed with Title IY-C funds. The system 
is currently being augmented through state school improvement funds 
targeted to the development of a computer-managed Career Magnet School 
(CMS) program. CMS seeks to align student career interests with 
competencies with specific courses at Site A in which they may be 
obtained. Over the past year, teams of teachers have been developing 
the list of skills associated with given courses in the school and an 
analogous list required for each career program. (Careers Magnet 
schools are first broken down into career paths such as computer 
technology and then further into programs such as computer technician, 
operator, programmer, or designer). The skill-to-course and 
skill-to-career matches are being computerized so that counseling 
staff can develop courses of study for students choosing particular 
careers and monitor their progress at obtaining prerequisite skills. 

CMS is one of two efforts recently instituted at Site A that are 
directed at career and academic decision making. The other, the 
Learning Resources Center (LRC), offers students and teachers academic 
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resources (materials, assistance) for remedial and advanced work. 
These two programs appear to be guided by a general concern that 
students do not have the necessary information and skills tn attain 
post-secondary education and careers they might warit. The implicit 
assumption is that by providing students with more information about 
career opportunities and their necessary prerequisites (CMS) on the 
one hand, and resources for remediating or enhancing their academic 
performance (LRC), on the other, that students will make better 
decisions about how to benefit from their high school experience and 
be better prepared for their future. 

Thus, Site A in the Valley Unified SchGOl District offers a 
setting with a wealth of relevant information already on file and a 
sophisticated system already in place to make that data accessible to 
at least some Site A personnel. It is a setting in which a few key 
figures had, before the advent of the current project, a keen interest 
in information for decision making. With these advantages, project 
staff have reasoned, if systemic evaluation cannot beconu. viable in a 
setting with these advantages, it probably cannot become viable 
anywhere. 

Aside from its promising technology, however, Site A and its 
district provide a typical suburban setting. There is reason to 
believe, therefore, that the kinds of issues, concerns, and 
enthusiasms that surface during the reality test here will be generic 
ones germane to circumstances in many other schools and districts. 
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The Systemic Evaluation Model 

The model or conceptualization of "systemic evaluation" that 1s 
1n trial at Site A has been outlined by S1rotn1k, Bursteln, and Thomas 
(1983). Among its key assumptions or features are the following: 

1. Outcome indices have limited value, beyond their immediate 
descriptive signal, for helping direct an agenda for school 
improvement. 

2. A necessary requisite is relevant information on the 
circumstances, activities and sentiments associated with 
schooling process. 

3. The criteria of relevance are based upon the perceived needs of 
the significant "actors" in the setting (e.g., administrators, 
teachers, students, parents) and the inherent value systems 
through which these perceptions are filtered. 

4. Information gathering as knowledge production has several crucial 
and interrelated features: 

a. It is operational ized with a mul ti -method approach to data 
collection (e.g., survey questionaire, interview, anecdotal 
and structured observation, document and archival records)* 

b. It is conceptualized and analyzed from a mul ti -level (e.g., 
individual class, school, district) perspective. 

c. It embraces multi-inquiry paradigms (e.g., empirical 
analytic, naturalistic/interpretive and critical-dialectic). 

5. Information as knowledge 1s not an end in itself but is, instead, 
a catalyst for evaluative discourse and action; systemic 
evaluation must, therefore, be legitimized as a natural and on- 
going part of the daily work life of those for whom the knowledge 
is to be relevant. 



9 

ERIC 



Underlying these principles is a view of the school as a cultural/ 

ecological system. Renewal of that system comes about ideally, the 

model suggests, through: 

a process by which the circumstances, activities, 
and meanings [of the school as a system] come to 
be understood and acted upon by people to whonfTt 
is relevant. L the renewl process] is people 
actively and continuously engaged in the systemic 
and rigorous deliberation over any and all 
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information seen to be potentially relevant to 
school improvement (Sirotnik, Bursteln, & Thomas, 
1983, p- 35). 

As all the foregoing should indicate, the model or 

conceptualization is not a blueprint or recipe for what to do, but a 

set of principles that can guide local developmental efforts. Thus, 

the "reality test," or feasibility study, was undertaken to learn what 

happens when educators in a school environment actually attempt to 

follow those principles toward development of sytemic evaluation. 

The Reality Test to Date 

The systemic evaluation field test is being carried out by the 
Center for the Study of Evaluation in collaboration with the 
Laboratory in School and Community Relations, both of UCLA. 

While examining the viability of the systemic evaluation concept 
is the broad goal of reality testing at Site A, there are more 
specific objectives as well. In particular, the project strives to 
explore and work toward resolving methodological issues that arise. 
In view of the nature of the model, two types of methodological issues 
are of central interest. One type can be called technical; the other, 
social . 

Technical issues include problems, concerns, and solution 
strategies that arise in developing methods to generate multi-method, 
multi-level data; to integrate that data technologically (e.g., in 
computer files) such that it can address locally relevant concerns; 
and to display that data in ways that different user groups find 
understandable and utilizable. {A paper that accompanies this one 
addresses so.ne of the technical issues that have emerged so far). 
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Social issues include the problems, concerns, and solution 
strategies that surface tn developing social organizational 
arrangements ~ arrangements for identifying "significant actors" in 
the setting; structures through which these actors 'can articulate 
their information needs and viewpoints on the formatting of 
information; forums and procedures that facilitate consultation and - 
use of the information; and arrangements for maintaining the entire 
evaluation-and-renewal system. 

By identifying and beginning to work through these issues during 
the field test, project staff hope to gain information that can help 
other schools and school districts develop their own versions of 
systemic evaluation n;ore smoothly and efficiently. 

To date, CSE project staff have worked closely with school and 
district personnel to carry through the initial stages of this reality 
test. A working group including the Site A principal, other school 
administrators, and a five teachers has met regularly with project 
staff. During the course of tJiese meetings, Site A personnel have 
begun to articulate their information interests and needs, to shape 
and revise a survey of student perceptions and attitudes, and to 
specify ways in which the student survey data and other information 
already in district files can be integrated and displayed for maximum 
utility. Three reporting formats have evolved. A "Student-At- 
A-Glance" form (including survey data, grade point average, attendance 
and test-score data on each learner) has been designed to accompany 
the class list each teacher receives at the beginning of a semester. 
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A "Class-At-A-Glance" form for each class provides graphic summaries 
of each class 1 perferences among different types of learning 
activities, as well as their overall feelings about the class 
subject. Finally, a "School -At-A-Gl ance 11 form highlights in graphs 
and prose a set of issues that the working group deems worthy of 
consideration by the entire faculty and staff. 

At the district level, project staff have examined the current 
district information files and have begun to develop (with district 
data-processing staff) ways of Integrating the data 1n those files 
with the new survey data. Negotiations among project staff, district 
administrators, data processing personnel, and the school principal 
have been aimed at securing continued district support for the reality 
test effort and for its maintenance when the field test ends. 
District officials have reaffirmed commitment to the project, and they 
have become increasingly interested in its data-Integration and 
data-reporting dimensions . 

At present, the project has several immediate goals. One is to 
refine the "At-A-Glance" forms and present them to all school faculty 
with actual data for their classes. A second goal is to present 
issues summarized in the 11 School -At -A-Glance" report to the entire 
faculty and staff 1n a forum which allows for their initial 
discussion. A third goal is to complete revisions of a teacher survey 
form, administer 1t to the whole school faculty, and integrate those 
data with information from the student survey and extant district 
information in ways that school staff believe will be helpful. These 
goals will entail continued cooperation with the District office, 
especially the District's data processing group. 

ERIC 
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And as these steps unfold, project staff will gather new Insights into 
the technical and social nethods that can facilitate the realization 
of systemic evaluation. 

Throughout the project, one staff member has tieen documenting the 
course of event — contextual features, trends in thinking, 
meeting-by -meeting discussions, etc. — in a nearly verbatim way* The 
600 pages or so of field notes collected this far have allowed UCLA 
staff to reflect on the direction of events over time. Ultimately, 
this case study will provide a record of what has been learned. 

The project also gains perspective on events in the field-test 
setting through recurrent meetings with another CSE effort (the 
Management of Instructional Information Systems, or MIIS, project), 
which has been studying and continues to investigate information 
systems in other school districts. 

This brief sketch of the reality test to date provides background 
for the description and discussion of some key contextual issues that 
have emerged so far. As that discussion proceeds, there will be 
references to and elaboration upon this outline of past and present 
activities. 

ISSUES AND OBSERVATIONS 

Four general features of the district and school context are 
described below, together with their implications for the development 
of a comprehensive information system at Site A. These are (1) the 
current status of the extant district-school information system; 
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(2) Site A teachers 1 clinical perspective on information and the 
features of school social organization that support that perspective; 

(3) faculty-District relations as they impinge on the development of 
systemic evaluation; and (4) the nature of District support for the 
Site A - UCLA project. While these topics are treated separately in 
the pages below, it will soon become clear that the phenomena 
discussed under each heading are quite interdependent. 

The Current Status of the Valley-Site A Information System 

As noted above, Site A has and is continuing to develop a 
computerized student-information management system. Referred to as 
CASA (Computerized Accountability for Student Achievement), this 
system was in place before the systemic evaluation reality test 
began. One might imagine, then, that CASA served to break ground for 
the development of systemic evaluation at Site A: that it 
familiarized Site A staff members with the advantages of easily 
accessible information for instructional planning and decision making; 
that it generated enthusiasm for and sophistication about information 
systems and their use; or that it led to technological and 
organizational procedures, upon which systemic evaluation could build, 
for accessing and using information. In fact, however, the presence 
of CASA has had little discernible impact to date on Site A's progress 
toward comprehensive, systemic evaluation. A brief review of CASA's 
history and current status reveals why; it also underscores some of 
the conditions necessary for instructional inrormation systems to be 
useful and utilized. 
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Like many educational information systems (Bank & Williams, 
1982), CASA emerged from the interest and enthusiasm of a very small 
group. One Site A assistant principal championed the idea of 
accessing the District's huge data base for Instructional planning at 
the school level* She took the initiative 1n CASA's creation. Later, 
as the Career Magnet School (CMS) concept evolved, this same 
administrator saw CASA's potential for facilitating 1t. Together with 
the Site A improvement program (SIP) coordinator, she wrote the plans 
that secured state SIP funds for linking CASA with CMS. (Refer to 
page 4 above for details.) A few others in the high school's central 
office, counselors and administrators, offered their support and 
suggestions as the present CASA system was developed. Evidence 
suggests that no faculty members were involved. 

Thus, CASA was in no sense a response to a school wide Interest in 
instructional information systems, in no sense a product of schoolwide 
demand for better or more easily accessible information about 
students. Among Site A's faculty and staff in general, no such 
interest or demand was expressed. Furthermore, with one exception, 
the few central office personnel who promoted and developed CASA did 
not seem to think of it as a schoolwide innovation. Certainly, there 
was no schoolwide planning to inform Its design. 

As it was originally intended to be, CASA has been used primarily 
by counselors and assistant principals. They draw upon it as they 
advise students on academic and career paths and as they handle 
referrals of students with individual problems. 
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CASA is also available to classroom teachers. Any member of the 
faculty can go at any time to the central office, where one of the y 
school's administrative assistants has been designated to process 
their CASA requests. Through her, they can obtain* information on 
individual students or class groups in any of a wide variety of 
standard report formats. (For a list of some of the data available 
through CASA, see page 4 above.) Nevertheless, all the evidence 
available reveals that very few teachers have made use of this 
opportunity. The following excerpts from the project's case study 
field notes document, and suggest some of the reasons for this 
phenomenon. 

NOTE #1 (February 22, 1984) . The first working-group meeting of the 
new project year is under way. Five teachers, an assistant principal, 
the school improvement coordinator, and the administrative assistant 
who coordinates CASA requests are present, together with the UCLA 
staff. Discussion has been following the first three items on the 
agenda prepared for the session by UCLA: "(1) Identify the kinds of 
information teachers, counselors, and building adminislrators view to 
be useful for their work*.. (2) Identify what specific problems (at 
any level) Sits A staff would expect the information system to help 
them address; (3) Ascertain level of understanding of the computerized 
information currently available to the school staff... 11 The teachers 
present have mentioned many kinds of information they feel would be 
beneficial to them: whether students in their classes have course 
prerequisites, students 1 past grades in their departments' courses, 
students' reading level, students' writing ability, and students' 
vocational goals. At this point, UCLA project co-director Leigh 
Burstein observes that nearly all this information is currently 
available to teachers on one of the standard CASA report forms known 
by the acronym ARF (Activities Referral Form). A foreign language 
teacher in the group responds, "I didn't know we could get that." 
Other Site A faculty 1n the group murmur their assent. Leigh Burstein 
describes the Activities Referral Form in more detail. The foreign 
language teacher asks, "Now why haven't we heard about that. That 
sounds very good." One or two other Site A staff members agree. A 
member of Site A's English Department replies, "I think we did hear 
something about that. Wasn't there — I think there was, uh, a 
presentation in a faculty meeting last spring about it." Someone else 
from Site A adds, "Oh, I remember that!" The Math Department 
chairperson comments with a chuckle, "It certainly made a strong 
impression, didn't 1t?" 
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NOTE #2 (February 29, 1984) . The second working-group meeting 1s in 
progress .with the same group members as 1n the first session a week 
earlier. The principal, however, 1s 1n attendance this time. Site A 
staff members 1n the working group havs produced lists "for Items to 
be printed on one sheet of Information, to be distributed to teachers 
one time per semester (one sheet per class)," as one group mender has 
labeled her 11st. T he suggestions on each 11st, which represent group 
members 1 own Ideas and those of colleagues with whom they have spoken 
during the past week, are now under discussion. As the foreign 
language teacher finishes presenting her suggestions, the following 
exchange ensues: 

Health Teacher : "It seems to me, urn, Isn't this just 

computerizing the cum. folder and Information card?" 

Hath Department Chairperson : "Well, it seems to me part of the 

problem is that all tne information people want Isn't 1n one 

place. It's 1n several locations. This would bring 1t 

together." 

Health Teacher : "Well I don't know that. I don't know where all this 
information is. I don't even know what questions to ask to get the 
Information I want." 

Math Dept. Chair : "You don't know whether to ask for an ARF or a 
RALPH or>.. (general laughter)* I'm not sure I do, either. 
Health Teacher : "That's 1t. That's It exactly." 
Later on 1n the meeting, discussion again returns to the ARF, or 
Activities Referral Form. UCLA staff have distributed an example of 
one ARF to the group. 

Leigh Burstein (UCLA) : "So far ARF 1s holding up pretty well, this 
information, the information we've been talking about, 1s in there. 
But the display issue will show up, because you want lots of 
information and it's not going to be easy to get 1t all clearly on one 
page." 

Ken Sirotnik (UCLA) : "This format [holding up the ARF] is too dense 

already. That may be why 1t Isn't used." 

Assistant Principal : "Well, It's quite new, too." 

English Teacher : ^No, he's right [holding up her copy of the ARF.] 

And it has to be 1n English . With all these numbers and codes, what 

have you ~ 1t won't work'ltke this." 

Several others from Site A's staff nod and vocalize their assent. 



HOTE #3 (Hay 23, 1984) , A student survey developed by the systemic 
evaluation working group 1s about to be administered in classrooms 
throughout Site A. To prepare the way for this, UCLA staff are 
spending the day meeting with small groups of Site A" faculty. During 
each period of the school day, those teachers with no assigned class 
meet with UCLA staff 1n a corner of the Learning Resource Center. 
There, Ken Sirotnik explains the history and purposes of the sytemic 
evaluation effort 1n general and the student survey 1n particular. He 
also answers teachers' questions, listens to their reactions, and 
outlines procedures for administering the student survey. Nearly 
every teacher 1n the school, plus some of the counselors and assistant 
principals, makes an appearance in one of the six sessions. In all 
but one of the six, there is considerable discussion and the exchanges 
are (1n the jargon of international diplomacy) "frank." 
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Throughout the entire day, despite the fact that many of 
Sirotnik' s remarks 1n each session could easily "cue 11 teachers 1 to 
think about CASA, only one teacher makes any reference to the CASA 
system, ARF, or any currently available information on students. 
Ken Sirotnik (during Period 4, finishing his Initial explanation of 
the project and the rationale behind the student survey): "So, we 
will report the information back to you ; and we may do so in several 
"STfferent ways that you can use, depending on what your colleagues who 
are working with us suggest. But I guess before I go on, now would be 
a good time to talk about issues £ou want to raise, and to hear your 
comments." 

Teacher : "You should know that we don't have a good record on getting 
information back. I was looking for correlations for ny class on tha\, 
[CASA] request form from [the administrative assistant who handles 
CASA requests]. I haven't gotten it yet. I think it's been over a 
week." 

Sirotnik: "Well, there may have been problems there, and that's one 
of the things we're working to avoid..." 

Teacher : "OK. I'm just pointing out that what we have is — it's not 
accessible now." 

These incidents strongly suggest that the CASA information system 
is neither well known now widely used among faculty at Site A. They 
give life and support to a comment dropped casually during the course 
of one meeting (with District personnel, on February 8, 1984) by the 
Site A administrative assistant who handles CASA requests. "We 
haven't been getting a lot of requests from teachers," she said. 
Indeed, the notes above suggest that this remark is something of an 
understatement. 

Faculty members in the Site A-UCLA working group — people who 
demonstrably have a special interest in information that can help them 
in their work, who care enough about obtaining such information to 
give their time to the project, and who volunteer that they would like 
to have some of the information now available in the CASA system — 
could not recall even hearing about CASA's availability. In the 
course of meetings with small groups of teachers, meetings in which 
each time teachers heard a description of a project very similar to 
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CASA in some of its goals and paticular features, no one asked, "Don't 
we have something very similar to this?" or H How is what you're 
talking about different than what we can get now?" Surely they would 
have done so, amidst the frank discussion that occurred, if they were 
aware of CASA. Given many teachers 1 expressed concern with the time 
costs and ultimate value of "one more survey," ttrs seems especially 
likely.* Finally, we have the administrative assistant's report 
(confirmed by District data processing personnel) that few teachers 
have called for CASA reports. All of this indicates why the presence 
of CASA has done little to pave the way for the development of 
systemic evaluation at Site A. CASA has, by and large, remained 
extrinsic to the professional lives of nearly all of Site A's 
faculty. 

It is not difficult to account for this state of affairs. Pieces 
of an explanation are threaded through the brief history and 
field-note excerpts provided above. 

First, and probably most importantly, nothing was done at Site A 
to build teachers' investment in CASA or to assure that CASA 
information and report formats responded to teachers' needs. There 
were, as we have seen, "good reasons" for this. CASA was not 
construed as a schoolwide organizational innovation. It was developed 
as a convenience, a technological short cut to facilitate central 
office and counseling functions. Teachers were told about it after 
the fact, rather than involved in its development. (Furthermore, 
within the flow of information that teachers constantly encounter in 
faculty meetings, this "telling" was evidentally not an especially 
salient event.) Thus, all the recent work on innovation and 
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dissemination in educational environments work captured in such 
buzz words as "stakeholders," "ownership," and "mutual adaptation" — 
helps to explain why CASA has not "caught on" throughout Site A. 

Second, there 1s evidence that the formal features of CASA (as 
opposed to its content) do not strike teachers as immediately 
appealing^ Working-group members, viewing the ARF report apparently 
for the first time, found 1t unattractive and hard to read (Note #2 
above). At least one teacher who requested CASA data found that he 
did not receive it promptly. Such shortcomings as these can be 
traced, at least 1n part, to the absence of faculty participation 1n 
CASA's development. They may also be factors in the limited teacher 
use of CASA, 

A third factor, not addressed in this account so far, may also 
have contributed to CASA's general confinement to the central office 
and to counseling tasks in particular. Teachers at Site A seem to 
take for granted that the functions of counselors and the functions of 
classroom teachers are generally quite separate. Thus, some teachers 
seem to feel that teachers need no more information about students to 
perform their roles. As one put it after hearing Sirotnik's expla- 
nation of the project in the small group meetings (Note #3 above): 

I don't claim to understand all this but it seems 
idealistic. This is qolng to be a great tool for 
counselors, but I don t see how I can use it as a 
teacher. 

What is more, teachers on more than one occasion have expressed 
frustration with the operations of the counseling office. Near the 
very outset of the working-group meeting described above in Note #1, 
for instance, as teachers were voicing some of their information needs 
off-the-cuff, one teacher said: 
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Let's get specific here. There's some things 
going on in the counseling office that are just 
very hard to understand. For instance, I teach 
social psychology, which is supposed to be a 
required course. But there is a procedure to 
waive the requirement. I'm interested in whether 
the kids who 1 re having it waived aren't the ones 
who really need that course. How can I get that 
information? I assume they have It in counseling, 
but how can I get it? Then, I get kids who're 
actually i£ the class who're on the waiver list... 

This remark set off a chain of anecdotes from other faculty in the 

working group about anomalies in course assignments and counseling 

decisions. The episode only concluded when the original speaker 

quoted above suggested that an information system should "kick out 

anything unusual, like the IRS computer does with tax returns, because 

obviously they're not doing this down there [in the counseling 

office]." To this, there was general agreement in the group. 

These and similar events indicate that in Site A's informal 

social structure, a boundary that is only semi-permeable marks off 

teachers and their concerns from counselors and theirs. This 

boundary, it appears, is reinforced by some strong teacher feelings 

about the quality of counselors' work. In the experience of this 

author, such a boundary is hardly unique to Site A; similar divisions 

exist in many American high schools. Nevertheless, its apparent 

presence at Site A may well have truncated the teacher- counselor 

communication through which information about CASA and its benefits 

might otherwise have been shared. The informal networks for such 

communication seem to be absent, and many teachers appear to believe 

that what is good for counselors is not necessarily advantageous for 

teachers. 
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This analysis of the history and current status of Site A's 
current information system provides some insights into the social 
milieu that surrounds the development of systemic evaluation at Site 
A. Principally, it should make clear that despite the existence of a 
relatively sophisticated information system at Site A, most Site A 
staff are neither enthusiastic nor sophisticated about instructional 
information or information systems. Furthermore, it should help to 
demonstrate that the social -organizational structures implicit in the 
systemic evaluation mode*! described earlier have not yet been built at 
Site A. 

Viewed in broader perspective, the account in this section begins 
to show that for all practical purposes, there is nothing special 
about the current status of information and information use at Site A. 
Most districts and schools do much less than they could with the 
evaluation and assessment data they have on hand (e.g., Bank & 
Williams, 1981; Dorr-Bremme, Herman & Doherty, 1983). Those that" 
routinely and systematically link it with instructional planning or 
school improvement are the exceptions. 

The next section of this paper explores how teachers at Site A 
think about information and their information needs under these, very 
usual, curcumstances. It also suggests how some of Site A's 
organizational arrangements support these ways of thinking. 



The Clinical Perspective and School Social Organization 

The concept of systemic evaluation that is generally guiding the 
Site A reality test (as outlined above, pages 6 to 7) places 
considerable emphasis on the social uses of information. 
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Information use by individual teachers and others is certainly 
included as part of the evaluation and renewal process. Nevertheless, 
information is viewed expecially as a "catalyst for evaluative 
discourse and action" (Sirotnik, Burstein, & Thomas, 1983, p« 4). 
The renewal process is construed as "systematic and rigorous 
deliberation over any and all information seen to be potentially 
relevant to school improvement" (p. 35). Such statements as these not 
only call attention to the importance of the social uses of 
information in the systemic evaluation model: they also reflect its 
commitment to a holistic renewal process. The school is viewed here 
as a cultural/ ecological system. Its parts are conceived to be 
interdependent. Thus, it is ideally the entire system upon which 
renewal efforts focus. This means that information use toward renewal 
is necessarily a social process, in which all relevant actors engage. 
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Especially in light of these ideals, it is important to note that 
Site A teachers do not routinely or easily think of information in 
terms of its social uses. Rather, they tend to approach information 
and their information needs from a clinical perspective. 

Two hallmarks of the clinical perspective, according to 

sociologists of applied knowledge, are its orientation toward action 

and its emphasis upon the individual case* Elaborating on these 

points, Homans (1950) explained: 

Clinical science is what a doctor uses at his 
patient's bedside. There, the doctor cannot 
afford to leave out of account anything in the 
patient 1 s condition that he can see or test. It 
may be the clue to the complex... In action we 
must always be clinical. Analytic science is for 
understanding but not for action. 
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Noting with Homans that the aim of the clinical practitioner "is not 
knowledge but action," Friedson (1970) adds that "the clinician is 
prone in time to trust his owr personal first-hand experience" and to 
be "particularistic," stressing the uniqueness of each case to be 
treated. The "clinical rationality," Friedson (1970, p. 171) 
concludes, "is particularized and technical: it is a method of sorting 
the enormous mass of concrete data confronting [the practitioner] In 
individual cases." 

It is the clinical orientation as defined here that character- 
izes the thinking of Site A teachers participating in the systemic 
evaluation reality test* Their central interest is particularistic- 
They want to know primarily about "this student" or (secondarily) 
"this class"; rarely do they manifest spontaneous interest in knowing 
about the students in "this school," or even those in a given 
department or program. They require that information be relevant to 
action, recurrently asking "What can I do with that?" and declining to 
gather information because "I can't do anything with it." In 
particular, they seek information that supplements, and helps them 
sort and clarify, the plethora of personal, first-hand information 
they gain about particular students and class groups as they interact 
with them. 

The project to date has gathered a wealth of data to substantiate 
these generalizations. Only a small portion of it will be reviewed 
here. 

NOTE #4 (Febrary 22, £984) . Leigh Burstein (UCLA) has opened this 
first meeting to discuss "what information you'd like to have 
available" by underscoring the many levels at which data can be 
aggregated to address needs of different types. He mentions 
"information for or on specific programs" and calls attention to the 
importance of data for "constant monitoring at the school level — 
O . course enrollments, drop out rates." He expands on his belief that 

ERJC^g information at the school level can help in the "planning and design, 
in studying the impact of new programs you want to start.*. In my view 
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this 1s what an information system should do." Two hours pass and the 
qroup takes a break. Up to now, none of the teachers mentioned 
^ig^ptlnfbi^ SnSa >ike to have on th ei r ndi vidua ! 
students and classes. T h e assistant principa l, in a brief comment 
has expressed nteres t in "a graph that makes attendence very visible, 
§Tt Suld show the match with time of day, day of week, neighborhood 
grid. This would help In working on attendence with neighborhood 
organizations and elementary schools. .. c((iicc 
The group returns from the break and continues to discuss 
Information on Individual students and classes. As the 3:30 p.m. time 
forTawIng the meeting to a close arrives, Bursteln again encourages 
the Site A participants to consider "measuring school climate Issues 
and the "on-going monitoring" functions of a comprehensive evaluation 
system. He assigns each working-group member to come Into the next 
meeMnq with a list of questions or Issues (1) the kinds of things 
[Information] you need for your students, classes, departments and so 
on or; (2) the kinds of things you'd want to collect at the school 
level on an on-going basis." The meeting ends with no further 
discussion of school-level information. 

Despite considerable prompting to consider other levels of data, 
then, teachers in this initial meeting framed their discussion of 
information needs exclusively in clinical terms. Their interest were 
in data on individual students and particular classes. And despite 
Burstein's assignment 1n preparation for the next meeting a week 
later, that meeting too focused almost exclusively on clinical 
information needs. 

NOTE #5 (Februa ry 29 1984) . The meeting opens with a discussion of 
the forelqn language teacher's list of desired Information. (Refer to 
Note #2 for context.) The 11st focusses on Individual student 
characteristics. Ken Sirotnlk (UCLA) then asks for other ideas. 
Social Studies Te acher : "I was talking to the people in my 
department, and tne most important thing people want to be able to do 
is to see what their class 1s like, so they can group kids for 
cooperative learning, so they don't assign work that s beyond the 
kids' level, things like that." 

K en Sirotnik : "There's not some other information that you, as a 
teacher, tmnk 1s more valuable...?" +u a +i c 
Social Studies Teacher : "Sure, there's lots of information that s 
valuable. And sure, 1 can wait for the class to begin and see the 
kids work, and then I get that Information. But what they want, what 
we want 1s for the beginning of the semester, when you don t know your 

Sirotnik : "Would they want students' GPA [grade point average] 1n the 

general area?" , ^ ^ 

Social Studies Teacher : "No, people didn't want to get that 
specific. What you need to know 1s what level are kids reading a*,? 
What are their comprehension skills? How well can they write? 
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English Teacher ; "Yeah, just a rough picture. You can modify it once 
you start working with them." 

Assistant Principal ; "You wouldn't want their GPA in the general 
area, as Yen was suggesting?" „„* A ** -tn 

Social S tudies Teacher : "No, but I'd like to have their grades in 
particular Lnglish classes. That alerts me to the kinds of success 
they have. It tells me not only how well they're doing but the kinds 
of strengths and weaknesses." ' 

A few moments later, the foreign language and social studies 
teachers turn to explaining some of the value of background data on 
idividual students. . . j _ 

Social Studies Teach er; "What we need is information that lets us 
respond to the kid who" says, when you give them the assignment, I 
can't read five pages.' you want to be able to go to your list La 
single page with information on each student in the class] and say, 
'That's not what this tells me. You're reading scores show you can do 

this...'" P . . «.„ 

Foreign Langua ge Teacher ; "Right. 'And it says here [gesturing to 
imaginary Information sheet] you have no job; you're in a college prep 
program, so I don't see a problem.'" 

Still further on in the meeting, UCLA staff try to turn 
discussion away from information on individual students and classes 
and toward consideration of school-level data needs. Sirotnik, for 
instance, argues that information on students' preferred learning 
methods and strategies" would need to be "content-free if it s to be 
useful schoolwide." Three teachers immediately respond that las one 
out it) "you can't ignore the subject matter if this is going to help 
us plan our classes." Burstein suggests that you could bank questions 
on instructional practices as part of an on-going effort to track the 
health of the school." As an example of such questions, he points to 
a Study Of Schooling survey the group is using as a stimulus for 
ideas. The agree-disagree questions listed there include such items 
as "The teacher gives me too much work to do in this class ; Students 
know the goals of this class"; and "The teacher tells us how to 
correct mistakes in our work." Burstein explains that these could be 
asked about "the teachers in this school in general," instead of about 
particular teachers in particular classes, as in the original. Site A 
teachers reply that this information would be seen as too 
threatening" if gathered about individual teachers and that it 
wouldn't be useful to anyone" if students answered about teachers in 
general. Teachers fears are discussed, and the meeting ultimately 
ends with no further discussion of school- or program-level 
information. 

These transactions demonstrate the persistence of teachers' 
clinical thinking. As in Note #4 above, the Site A teachers in the 
working-group and the colleagues with whom they spoke simply could not 
independently generate suggestions for data that would be worthwhile 
at the school level. Here too it is evident that they want 
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information that can help them make sense of and respond to Individual 
cases: information for planning their class; Information for respond- 
ing to the student who says "I can't read five pages"; and so on. 
Furthermore, they want this particularistic information at the 
beginning of the semester. Once teaching and learning are under way, 
they will have additional Information on students from their own 
experiences with them. This Information will allow them (as one 
teacher maintains) to "modify" the general view of the individual 
learner that they can obtain from test-score and GPA data. Thus, the 
clinicians' tendency to trust personal, first-hand knowledge comes 
through, as well, in these conversational exchanges. 

The clinician's action orientation 1s evident 1n all the above, 
but it is especially apparent in the following. 

NOTE #6 (April 3, 1984) . The group is working Its way through 
student attitude surveys originally used in the Study Of Schooling 
research, selecting Items and issues that seem likely to generate use- 
ful information that can be tied to the District's huge data file. 
About twenty minutes have gone by when the group turns to a set of 
agree-disagree questions headed, "Relation to Other Students.' Among 
the six Items are such statements as "I'm popular with kids my own 
age" and "It's hard for me to make friends." Introducing their con- 
sideration, Ken Slrotnik (UCLA) explains that they "cluster to yield a 
score which you could call 'self-concept toward others.'" A debate 
erupts about who would use this Information. The foreign language and 
health teachers maintain they're not interested. "I wouldn't have any 
need for that," says the former. Leigh 8urste1n (UCLA) argues that 
this data could shed light on the school -wide attendence problems 
"you've all been concerned with." He also notes it could be used 1n 
an on-going monitoring of the health and climate of the school. Ken 
Sirotnlk adds, "The question here may be not so much what you're going 
to do with this in your class, but what's a whole faculty going to do 
if they find many stUdehts have a low self-concept, there's attendence 
problems at the same time. Shouldn't they know that?" 
Health Teacher : "Look, what I'm saying is who uses this? If I were 
starting a school, I might want know this, but who 1s there right 

now? " 

Social Studies Teacher : "I think the counselors might want it. Id 
want it if I were a counselor." 

Foreign Language Teacher : Even though I didn't choose these, I have 
no objection to asking them. 
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Math Dept. Chair : "You could use this for longitudinal monitoring, 
though, like Leigh has said, couldn't you?" 
Burstein : "I hope so." 

HealthTeacher : "That seems Hke we're just collecting information 
for the sake of collecting Information." 

English Teacher : "No, I think this could really help with the 'Track 
A kids, in confluent education. I'd want to know how nty students 
feel about themselves." 
Discussion passes on without a consensus. 

Soon attention turns to a set of questions that elicit students' 
view of the quality of the school's "physical plant." Everyone agrees 
this is within the administration's purview. The issue of whether to 
Include these is resolved when one teacher says, "Let [the assistant 
principal] and [the principal] decide 1f they want it." 

Near the end of the meeting, a set of questions about why 
students elect the classes they do 1s examined. Several teachers 
point out that since the school administration has just made decision 
to limit students' choice of classes, this Information 1s irrelevant. 
"We can't do anything about this," the social studies teacher 
reasons. Burstein again raises the importance of considering the 
value of this Information 1n a longitudinal sense," but the teachers 
end up rejecting the items as useless. 

As noted above, the views expressed in this excerpt highlight the 
action orientation of the clinical perspective, as manifested in the 
concerns of Site A teachers. Together with the other field notes 
excerpted and transcribed here, this helps to document that Site A 
teachers do indeed approach Information and their information needs 
from the clinical perspective. 

There is nothing especially surprising in this finding. Teachers 
across the nation seem to adopt a clinical stance in seeking, 
interpreting, and using data about students (e.g., Dorr-Bremme, 
1983). Nevertheless, the account presented here suggests that a 
substantial gap exists between teachers' routine ways of thinking 
about Information, on the one hand, and, on the other, the way 
systemic evaluation principles posit that they should. This gap is 
currently an Important part of the context in the development of 
systemic evaluation at Site A. What is more, 1f the national data 
just referenced are accurate, it promises to be a key contextual 
factor in. many schools and districts. 
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While Site A teachers perspective on Information is a particular 
* kind of perspective (a clinical one), it also has certain generic 

features. Like members of other organizations, teachers at Site A 
are interested 1n Information that has "theoretical or practical 
Import for organizationally relevant purposes and routines 11 
(Garfinkel, 1967, p. 191). It simply happens that, given the soda! 
organization of Site A, the only organizationally relevant purposes 
and routines for teachers there are clinical 1n nature, I.e., taking 
action toward Individual students and class groups. At present, the 
school maintains no regular organizational structures that bring staff 
members together and empower them to discuss and resolve common 
concerns. 

As most high schools in the United States are, Site A is 
organized into various academic departments. Department meetings 
occur, but they apparently do not consitute settings for dialogue and 
conjoint decision making. When Bursteln suggested that some type of 
survey data might help departments plan their currlcular emphases, 
faculty 1n the working group rejected the notion immediately. One 
teacher explained that "everyone sort of sidesteps disagreements over 
teaching methods and philosophy and things" during department 
meetings. Another added that departments meet infrequently and 
usually deal only with what courses individual members of the 
department want to teach, what books they want to order, and similar 
routine tasks. In another working group session, Burstein asked 
whether "your departments" could use information on students 1 
perceptions of instructional practices. After some initial confusion 
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about what Burstein had in mind {several Site A participants asked in 
apparent disbelief, "Why?" and "For what?!"), one teacher answered, 
"No, this would be seen as threatening." The matter rested there. 

Site A, as noted 1n passng earlier, participates 1n the 
California School Improvement Program (SIP). SIP guidelines require 
schools to assemble a school site council which Includes the principal 
and elected representatives of various constituencies: teachers, 
other staff (e.g., counselors, non-cert1f1ed personnel), parents and 
other community members, and students. According to SIP provisions, 
the site council has responsibility for assessing schoolwide needs, 
developing improvement objectives, planning activities to meet them, 
and evaluating the results of these efforts. SIP Schools are 
encouraged to engage in comprehensive planning and to use the planning 
process as a catalyst to or motivator for dialogue and involvement. 
Some schools do so, but many treat the planning process merely as a 
hurdle that must be jumped in order to procure additional state monies 
(Dorr-Bremme, et al., 1979). Site A seems to fall in the latter 
category. Several discussions of Site A's SIP program during 
working-group sessions suggested that a few administrative leaders 
have primary responsibility for SIP plans. Thus, the SIP site council 
does not appear to provide a forum for substantial teacher involvement 
in schoolwide planning and decision making. Similarly, other Site A 
instruction-related programs (the Career Magnet Schools and Learning 
Resource Center programs, for instance) appear to be the artifacts and 
concern of a few key administrators. 
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More generally, role boundaries are relatively well defined at 
Site A. Teachers teach; counselors advise students, help them plan 
their programs, and deal with special problems; administrators set 
policy and concern themselves with schoolwide issues. That faculty 
members currently tend to accept and cooperate in sustaining these 
boundaries should be evident in some of the remarks quoted earlier. 
(Teachers referred the decisions on whether to gather certain student 
attitude data to counselors and administrators, for example. See Note 
#6.) 

It is not the case, then, that teachers are simply predisposed to 
see information and their information needs from a clinical 
viewpoint. Their clinical perspective is in fact supported by the 
organizational arrangements within which they operate each day. Those 
arrangements provide no occasion for using information socially; they 
generate no need to consult school -level information. Thus, when 
teachers consider the marginal utility of new information - when they 
implicitly and explicitly address the common organizational question, 
"Will it have been wortii the cost to gather this data?" — the 
criteria they employ and the decisions they make reflect the practical 
contingencies and exigencies they face on the job (c.f., Dorr-Bremme, 
1983; Garfinkel, 1967). They can use additional information to 
operate clinically in the classroom. They cannot use information for 
anything else. 

It follows from this analysis that Site A's organizational 
structure is every bit as important a contextual influence in the 
development of systemic evaluation there as teachers' clinical 
perspective is. Indeed, this analysis suggests that the two are 
ERJC interdependent. Other CSE/UCLA research supports this view, in 
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studies of testing-evaluation-instruction linkage systems in school 
districts, Bank and Williams (1983) found that organizational 
supports for information use and teachers' attitudes toward 
information tend to evolve together and be mutually supporting. 

Faculty - District Relations: Conflic t and Trust 

The current status of the CASA information system, teachers' 
clinical perspective on information, and the organizational 
arrangements that help sustain the latter are aspects of the school 
context that have already begun to influence the systemic evaluation 
field test at Site A. Now, attention turns to contextual factors 
which promise to influence the field test in the immediate future. 
One of these, discussed in this section, is the demeanor of the 
faculty at large toward the evolving systemic evaluation innovation. 

Recall that to date only a small number of Site A faculty and 
administrative staff members (the working group) have been intimately 
involved with the project. The remainder of the faculty has 
participated only tangentially in two ways. First, they attended 
small group meetings during which the systemic evaluation project's 
history and goals were described. (See Note #3 above.) Second, they 
administered a student survey during one class period. 

Soon, however, the faculty and staff at large will begin to play 
a much more important role in the project. On November 7, they will 
attend a three hour faculty meeting devoted exclusively to the 
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systemic evaluation project. During that meeting, they will review 
and react to the "Student-At-A-Glance" and "Class-At-A-Glance" forms 
developed by the Site A - UCLA working group. They will also examine 
four key student- survey findings incorporated on the 
"School -At -A-Glance" reporting form and begin to consider whether 
these merit schoolwide action. (These forms are described above on 
pages 8 and 9.) A teacher survey will be administered later in the 
year. And, assuming continued District support, all teachers will 
receive Student-At-A- Glance and CI ass-At-A-Glance data for each of 
their classes at the beginning of the second semester in the present 
school year. Faculty reactions to the systemic evaluation project as 
a whole will soon become extremely influential in the course the 
project takes. The responses of teachers throughout the school will 
also, of course, contribute significantly to the learnings that the 
reality test has been designed to engender. 

Under these circumstances, it is especially important to ask, 
"What is the present outlook of the faculty? How are they now 
inclined to view activities of the type that systemic evaluation, as 
developed at Site A, is likely to entail for them?" 

With respect to the kinds of information teachers in general are 
most likely to find interesting and relevant, there is every reason to 
believe that viewpoints of faculty members in the working group are 
likely to be broadly representative. This is also true of the formats 
in which they are most likely to find that information appealing and 
useful. As noted earlier, working-group teachers have tried to speak, 
at least in part, for the faculty at large. On at least one occasion, 

ERIC 35 



Dorr-Bremme 30 



they elicited the views of others in their departments (see Note #5 
above). Thus, the particularistic, clinically oriented information 
presented in the concise, Student-At-A- Glance and CI ass-At-A-61 ance 
forms seem to be reasonable "first draft" products to offer the 
facuHy as a whole for their review. 

At the same time, however, working-group teachers recurrently 
caution that their perspectives are not shared by all faculty 
members. Some, they point out, have no Interest at all in survey 
information on students 1 feelings and viewpoints. As one teacher 
explained during a working group session: 



I'm interested in what students Hke to do in 
class and what methods work best for them, but I 
don't think all faculty would be interested in 
this. Many of them would say, "when the bell 
rings, I'm going to close that door and teach what 
I want to and the way I want to based on what J_ 
think is best. 

On another occasion, during a working-group discussion on the value of 
gathering data on students 1 views of teachers 1 helpfulness, a 
different member of the working group opined, "There aren't probably 
ten people throughout the school at this time who care anything about 
that." 

Even such "hard" data as test scores showing students' reading 
level and writing ability, teachers in the working group warn, is not 
of universal relevance. 



If I were to ask the question, how many teachers 
will use this, I'd have to say, well a small 
number. You've got to remember there are 93 
teachers out there and in some areas — in PE, 
horticulture, and some others — they don't 
concern themselves with this stuff. 
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In short, working group teachers emphasize, as one might expect, 
that every faculty member will not be interested in a given type of 
information about students and some may not be interested in any 
student information at all. 

But the particular information preferences of Site A teachers are 
less important, at this point in the project's development, than their 
general stance toward the project and its data collection and 
reporting activities. To engage in dialogue with UCLA staff and Site 
A working group members, to be willing to give the project their time, 
Site A faculty and staff must first believe that some kind of data 
collection and reporting activity can be worthwhile. Put another way, 
they must be more-or-less convinced that the benefits of a project 
such as this one can_ be worth the costs — the time and energy they 
will have to invest in it. It appears that at present not all Site 
A's professional staff are. 

During their initial introduction to the project in small -group 
meetings on May 23, 1984, a notable minority of teachers voiced doubts 
that benefits would come from the systemic evaluation effort. Such 
doubts seem reasonable in light of the experiences they report, as the 
following quotations from participants in the May meetings reveal. 

- There were four surveys already this year. We 
never saw the results of any of them. 

- When [the new principal] was here [for a visit] 
last spring, he gave us a questionnaire. Has 
anyone seen the results of that? 

- You'll find your facing a negative atmosphere 
here. The teachers have been "statistized" to 
death. This [student questionaire] is just one 
more survey. There has been no follow up from the 
district on any of them yet. 
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- We gather a great deal of useless information here 
already... As long as the teachers is powerless to 
act on the information about students, then it all 
just disappears in quicksand. It's more 
frustrating than anything else if you can't do 
anything with 1t. 

- I know you [UCLA people] need this for your 
professional careers; It's nothing personal, but 
many teachers have had too much of ed. school 
people who do research and then run off and give 
boring lectures. And who does 1t help? 

While these remarks explicitly express skepticism about the 

benefits of any data collection enterprise, implicitly they also 

reveal concerns about costs. As yet another Site A teacher explained 

during the Hay 23 meetings: 



It's not just you, but you're one of 900 separate 
projects with demands... We've had Students 
Against Drunk Driving, substance and drug abuse 
projects, and each one got their time 1n my class. 

Stm others who spoke up in the small-group meetings. worried 

that the data would be Invalid, negative, and bad for the faculty. 

"You're going to get automatic reponses on this," one explained to the 

UCLA speakers. "The majority of kids will respond in very negative 

ways that they may not really feel." Said another, referencing 

changes 1n school policy and procedures under the new principal, "This 

1s a year of change and It's a bad time of the year, too. It 

guarantees negatives." Yet another argued: 



We have a history of being told the results of 
everything in a way that points out our 
weaknesses. The kids have just been taking tests, 
and regardless of the results, they'll find fault 
with us somehow. What they'll say about our way 
of giving tests will be negative. 
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"If this gets out," warned still another teacher, "it's going to just 
be more fodder for the press to condemn us with," 

Working-group teachers, who participated in the May small-group 
sessions with their colleagues, attempted to put all these negative 
initial reactions in context for the UCLA staff ♦ Faculty morale at 
Site A, they explained, was very low in general ♦ More specifically, 
working group teachers went on, many teachers were afraid that the 
data would be used against them. Recent events in the District were 
at the root of all this, they added. 

It seems that during the 1982-83 school year, the Valley Unified 
School District felt impelled to reduce the number of faculty it 
employed districtwide. Even some tenured teachers had to be let go. 
Teachers found the Districts "riffing" (or reduction in force) 
procedures highly unfair. Wi thin-district transfers that accompanied 
the staff reduction, they maintained, resulted in teacher assignments 
"that make no logical sense." 

Part of the "riffing" process involved the use of information. 
According to one teacher, the 153 district faculty members listed for 
possible lay-off, 

were all checked out. They [District officials] 
were looking at projector use. They figured that 
teachers who were Showing a lot of films weren't 
teaching. They considered people's academic 
qualifications to teach subjects, their classroom 
control, anything that would justify cutting 
them. So all this [student survey] information — 
well, there's soTie specific feelings of mistrust. 

Later on, in the 1983-84 school year, contract negotiations 

between the local teachers' association and the District reached 

impasse. Teachers felt not only that the District's firm salary offer 
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was unsatisfactory, bui it was extremely unreasonable. This situation 
aggravated the wounds opened by the reduction in force, leaving 
teachers feeling beleagured and unsupported. Problems that might 
otherwise have been interpreted by teachers as petty, bureaucratic 
inefficiencies came to be viewed as evidence of the District's 
disregard for their professional status and needs. 

Whether all the comments quoted above reflected this state of 
affairs is problematic. It is probably reasonable to assume that some 
did, as the working-group teachers maintained. Furthermore, other 
faculty members 1 comments in the May 23 meetings made these links 
explicit. 



NOTE #7 {May 23, 1984). The Period 2 small-group session is under 
way^ Ken SirotmK lutLA) has explained the project's aims and 
elicited reactions. Many are negative. 

Social Studies Teacher: "Part of the reaction you'll be seeing all 
day lies in the fact that we're in the classroom, we need supplies, 
materials, support of this kind and we're not getting it. 
Administrators are going off to meetings, intellectualizing about new 
educational ideas, but we can't get what we need to do our jobs." 
English Teacher: "I have a college prep class, British Literature. 
There are books we need to read, which I ordered ages ago. TTiey 
haven't arrived on time. How am I supposed to teach literature 
without books?" 

Social Studies Teacher: "The anthro. books I ordered in September for 
this semester haven't arrived yet." 

Second Social Studies Teacher: The same thing has happened with my 
global geography text. There~are the problems we face, the practical 
day-to-day things. What am I going to do with more information? 
English Teacher: You're dealing with a very embittered staff. 
There's conflict between the teachers and the Hill [the District 
office]. Teachers here have been mistreated." 
Later, in the sixth period session, a teacher echoed these themes. 
"Why should we get excited about information systems, "he asked 
rhetorically, "when the District can't even order me my books. This 
[student survey] will just tax an already over-taxed system." 

By rough count, 68 Site A faculty and professional staff meters 

attended the six small group sessions held on May 23 to introduce the 

systemic evaluation project and its student survey. Thirty three 
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(about 48%) took the floor to address their group and UCLA partici- 
pants 1n particular. And of those 33 who spoke, 19 {roughly 57% of 
the speakers and 28% of the staff 1n attendence) articulated one or 
more negative comments of the types quoted above. Nine others asked 
questions that can be construed as neutral 1n tone, e.g., "Are there 
standard procedures for administering [the student survey]? 11 ; "What 
should we do 1f the student doesn't know his CMS [Career Magnet 
School]?"; "When will you have the results back?" The remaining five 
speakers (Including two members of Site A - UCLA working group) 
offered comments which can be Interpreted as positive toward or 
supportive of the project 1n general or the student survey in 
particular, e.g., "Some of these questions look Interesting"; "I can 
vouch for Ken [S1rotn1k of UCLA]; I've worked with him before. He'll 
follow through." 

These simple counts should help to put the discussion 1n this 
section -- and the demeanor of those at Site A toward the project -- 
into clearer perspective. Initial responses toward the project were 
universally enthusiastic. Faculty attitudes toward systemic 
evaluation activities are, apparently, mixed. Previous experiences 
have generated skepticism about the value of data collection, as well 
as resentment toward yet another activity that takes time away from 
the central business of teaching and learning. The District adminis- 
tration and Its creations (and this project can be interpreted as one 
by Site A staff) are viewed with antipathy and suspicion by many. All 
of this provides a challenge for the development of systemic evalua- 
tion at Site A. It 1s a challenge that can be met only in part by 
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presenting the faculty and staff with relevant information at relevant 
times in appropriate formats. Equally important, if not in fact more 
fundamental, is building an environment of greater trust and 
cooperation in a social system where mistrust and antagonism have 
recently been more common. 

While this section has focused upon the initial attitudes of 
Site A faculty toward the systemic evaluation project and its reality 
test activities, it has shown some connections that appear to exist 
between those attitudes and the doings of the District 
administration. The next section examines the role of District 
leaders more explicitly and cites some reasons why their role is 
important in a school-based innovation. 

The Nature of District Support for the UCLA-Site A Project 

The last section began by suggesting that teachers 1 demeanor 
toward reality- test activities is one contextual feature that bears on 
the future of systemic evaluation at Site A High School. The nature 
of the District administration's support is another. In order to 
understand the present status of that support and what it portends for 
the project, however, a bit of background is necessary. Thus, the 
discussion below opens with a brief review of the school and District 
commitment to the systemic evaluation reality test. Then, it moves on 
to consider how and why the District's commitment to the project is 
likely to impinge upon systemic evaluation's future at Site A. 
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The systemic evaluation reality test has had consistent, strong 
support from the Site A administration. The new principal (who 
assumed his position 1n September of 1983 and 1s now 1n his second 
year) appears to be enthusiastic about the project's potential. He 
has taken the lead in securing release time for working-group teachers 
to participate in the project. He agreed to devote class time to the 
student survey and has scheduled the planned, three hour faculty 
meeting to examine the "At-A-Glance" reporting forms and survey 
results. Despite a very busy schedule, he has attended all but one or 
two working-group meetings and several meetings among UCLA staff and 
personnel 1n the District office. Furthermore, he has expressed 
interest in the student survey findings, made supqustions for how they 
can be useful schoolwlde, and cited them 1n support of several 
policies and decisions. The principal also has committed himself to 
involving the faculty as a whole in examining andacting on schoolwlde 
issues that project data help to make evident. All of this 
demonstrates his continuing support for the development of systemic 
evaluation at Site A. 

The assistant principal who championed the CASA system and acts 
as a member of the working-group 1s another key figure in the school 
administration's commitment to the project. A second assistant 
principal has become increasingly interested 1n systemic evaluation 
activities through recent months. 

In Its commitment to project activities, the school 
administration has been generally reinforced by administrators at the 
District level. The Superintendent of the Valley Unified School 
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District involved it in a network &f schools usually called The 
Partnership, sponsored by the UCLA Laboratory in School and Community 
Relations* It was through Partnership connections that the Valley 
District and Site A were identified as promising settings for the 
systemic evaluation reality test* The systemic evaluation concept 
appealed to the Superintendent, and he welcomed the project. 

The District's Assistant Superintendent for Instructional and 
Support Services is the Superintendent's designated contact for 
project issues. He has said on several occasions that building a 
comprehensive instructional information system "is something I've 
always wanted to do." With the Assistant Superintendent's approval, 
release time from the classroom for working-group teachers has been 
made possible, student surveys have been printed, survey data have 
been tabulated, and the District's information on students have been 
make available to UCLA staff. His approval of these activities 
testifies to District interest in and support of the systemic 
evaluation idea and reality-test activities. 

As the 1983-84 school year ended, the Assistant Superintendent 
affirmed the District's continuing "commitment" to the project. 
Ratifying that commitment, he promised approval of extra pay for 
teachers to join in a two-day working-group meeting in September of 
1984, just before the official opening of the new school year. 

There is reason to believe that the Assistant Superintendent's 
interest in the systemic evaluation effort is more than nominal. 
During a meeting on September 27, 1984 to review project progress and 
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products (the "At-A-Gl ance" forms and student survey data), he 

expressed enthusiasm with the forms. He became involved in an 

animated discussion of survey findings. He also suggested other, 

school -level data that would be helpful to the District and to school 

counselors. When the head of District data processing suggested that 

some of this information was already available, the Assistant 

Superintendent replied: 

I know you have it but the problem is to get it 
into a form like this , a form that's easy to read! 

He then went on enthusiastically to suggest various ways in which 

project designs and concepts could be adopted by the District for its 

information needs. 

The District's data processing personnel were equally affirmative 

in their reactions to the "At-A-Glance" forms and the information they 

contained. The director of the data processing unit, who often 

worries aloud about his need to keep information and produce 

information and produce reports "that nobody really wants," commented: 

The thing you [UCLA] guys did was go out and find 
what people wanted . That's what's important... 
This is, this is a pretty nice report! 

He added that the junior-high- school counselors "would like this" and 

was soon immersed in a discussion of the techical prerequisites for 

producing the At-A-Glance reports on a routine basis. 

Thus, key District personnel have manifested genuine interest in 

the substance and products of the joint Site A - UCLA project. They 

have commented with some enthusiasm on their potential for wider use 

in the District. Together with their limited-but-important financia 1 
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support (for survey duplication, data processing time, and 
substitutes to release working-group teachers from their classroom), 
this interest promises on-going commitment to the reality test, if not 
to systemic evaluation itself. 

Other events, however, blur the portrait of District support 
painted so far. First, the Assistant Superintendent equivocated on 
his commitment of extra pay for working-group teachers to attend the 
planned, early-September meetings. According to the Site A principal, 
he suggested in early August that the funds for this extra pay could 
come from a special state grant the school was hoping to win, but 
added that "we'll find some way" to compensate them for their 
additional work time. Later in August, a phone call to the principal 
revealed that the Associate Superintendent had not yet formally agreed 
to such compensation. The principal delayed inviting teachers to the 
two-day September meeting pending a District commitment to "compen- 
sation. Plans to expand the number of teachers in the working group 
were also delayed. In yet another phone call to the principal three 
days before the scheduled meeting in September, he reported, "I don't 
know about compensation. I think we have some money here in the 
school that will cover [the teachers'] lunch." Ultimately, the meet- 
ing took place, but for four hours instead of the scheduled two days. 
The working-group was not enlarged; only the administrators and four 
of the five faculty members who had been working with the group all 
along attended. Given the ambiguity surrounding the compensation 
issues, it seemed inappropriate to increase the group's size or to ask 
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teachers to donate more of their time in the days just before school 
opened. The state grant application (mentioned above) was not 
successful ♦ No District funds were forthcoming. 

Second, during the September 27, 1984 meeting cited earlier, the 
Assistant Superintendent maintained that "there's no extra money" for 
various District office work In support of systemic evaluation, UCLA 
staff and the Site A principal had explained their interest 1n 
developing and administering a teacher survey, using District data 
programmers to produce "Students-At-A-Glance " (and possibly "Class- 
At-A-Glance") forms for all Site A teachers 1 second semester classes, 
and continuing working-group meetings with ten faculty members rather 
than the present five. The Assistant Superintendent was most 
concerned, in the absence of "extra money," about the costs of release 
time for teachers and the level of effort District data programmers 
would need to invest. "My feeling 1s," he said, "we've invested time 
in this and we should go the next step, but we can't omit the fact 
that we don't have a lot of bucks." In the end, the Associate 
Superintendent requested the director of data processing to give him 
"an ideal of how much time this will take. If It's two months of two 
programmers' time, maybe we can't do it." He also directed the Site A 
principal "to come up with an estimate of the release time" for 
teachers that the project would require through the 1984-85 school 
year. "I'd like to go to the Cabinet with the whole package, but I 
need to tell them what 1t would cost." 

These time-cost estimates were subsequently provided, but by 
mid-October no response from the Assistant Superintendent on any of 
these matters had been received at Site A or UCLA. 
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Third, the ambiguous nature of District support for the project 
demonstrated by these events may be part of a larger pattern. In the 
veiw of several persons with whom UCLA project parrtici pants have 
spoken, the District administration has a tendency to let schools do 
whatever they wish, but to drop projects once the extra District 
monies or outside funds for those school-initiated projects are gone. 

In overview, then, District support continued to be quite solid 
through the 1983-84 school year. As the 1984-85 year opened and UCLA 
staff made a tactical decision to encourage the District to assume 
greater responsibility for systemic evaluacion, however, the nature 
and extent of that support became less clear. 

The foregoing account should demonstrate that a districts 1 sup- 
port for a school-level innovation is subject to on-going negotia- 
tion. As circumstances at the district level evolve, priorities 
change; new demands on district resources arise. Earlier "commit- 
ments" to particular projects need to be re-examined by district 
administrators in light of changing circumstances. At the same time, 
school personnel cannot always specify in advance exactly what a 
development project will require in terms of district support. UCLA 
participants were careful to detail what, in general, the project 
would require of the Valley Unified School District and Site A. 
Release time for teachers throughout the reality test was mentioned. 
So, too, were data processing time and clerical support (for duplicat- 
ing surveys, etc.). But until longer-term plans and particular data 
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collection and ;eport1ng procedures emerged from the developmental 
process, no one could say with precision just how many teachers would 
need to be released from class assignments for how many hours, how 
many pages would need to be duplicated by what deadlines, or how many 
District programmers would need to work for how long 1n support of the 
project. Thus, just as changing District political and economic 
circumstances open the door to renegotiation of levels of support for 
a given Innovation, so too does the evolution of needs at the school 
level 1n any formative or developmental project. There 1s nothing 
especially unusual, then, 1n the sequence of events described above. 

Considered in broader perspective, the nature of District support 
is likely to be a key contextual factor 1n the long-term maintenance 
of systemic evaluation at Site A.* The support and collaboration of 
both school and district leadership tends to be critical 1n thw 
maintenance of Innovative educational programs (e.g., Berman & 
McLaughlin, 1977). This holds true when the Innovation 1n question 1s 
an Instructional Information system. Bank and Williams (1981, 1983), 
for Instance, have studied a small number of school districts that 
have made unusual advances in Unking testing and evaluation data with 
instructional planning and decision making. In none of these cases 

*In the short run, UCLA resources through the Center for the Study 
of Evaluation from the National Institute of Education and from the 
Laboratory 1n School and Community Relations will sustain the reality 
test. Of course, the project's development activities will probably 
need to be adjusted 1f the District provides no financial support. 
There 1s, however, every likelihood that Valley will continue to bear 
the costs of at least some reality- test activities. 



49 



Dorr-Bremme 44 



9 

ERIC 



was the mere presence of relevant and readily utlllzable Information 
sufficient In Itself to sustain the links or guarantee the 
Information's use. In every district studied, there were one or two 
idea champions at the district level who took the lead in sustaining 
the linkage system. Each district devoted considerable resources to 
structures that supported the system. Most maintained on-going staff 
development which helped teachers learn how to Interpret and act on 
the available information in their everyday activities. All created 
(or capitalized upon extant) organizational arrangements within which 
school personnel were empowered to use the information in making 
choices among alternative educational policies and practices. In 
short, these exemplary districts reified their commitment to and 
support of instructional Information systems by Institutionalizing 
them in a network of mutually Interdependent and mutually sustaining 
activities carried out collaboratively in both school and district 
settings. 

Now, the systemic evaluation effort differs from those studied by 
Bank and Williams. The latter were district generated* and 
distrlctwide 1n scope; implementation moved "t;ip down," from district 
office to the schools. In contrast, the joint UCLA-Site A project 
tests a school -based, "ground-up" approach to trie development of 
information systems. At the school level, some of the functions Bank 
and Williams Identified as components of successful instructional 
information systems have begun to emerge. While UCLA participants 
were the original idea champions, the Site A principal, an assistant 
principal, and the cadre of working-group teachers have begun to 
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assume that function, (They have enthusiastically taken 
responsibility for the upcoming faculty meeting to present the project 
and Its products to the entire staff, for Instance.) Through 
working-group sessions, they have begun to learn how to think about 
Information and its uses in Instructional planning and decision 
making. 

Nevertheless, it seems likely that the Valley Unified School 
District will need to take at least some steps along the path marked 
out by Bank and Williams 1 findings if Site A is to maintain whatever 
information system UCLA helps the school develop. As noted in the 
last section, teachers at Site A are skeptical about the value of 
information. They define their relationship with the District 
administration as one of conflict, rather than collaboration. Thus, 
it seems unlikely that Site A faculty will give their time and energy 
to systemic evaluation without guarantees of District support. 
Furthermore, as the role of UCLA personnel gradually changes from one 
of initiation and leadership to one of support arid study, the 
District's 'commitment will become more critical. Even though Site A 
working-group members may assume leadership, they will need help and 
guidance from the District office. They will need, at a minimum, the 
resources required for data collection, processing, and reporting in 
relevant formats. They will also need time to consider and discuss 
school -level data; to act on it in ways they deem appropriate for 
school renewal; and to review, revise and (thus) maintain their 
information system. They will also need problem-sol ving advice on 
technical and social organizational issues. All of this, it now 
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seems, will have to come directly or indirectly from the District 
office. Given the District administration's history and present 
ambiguous demeanor, it remains problematic how much of this support 
the District will choose to provide. 

CONCLUSION 

This paper has discussed four key social contextual issues that 
have emerged during the early phases of the joint UCLA-Site A systemic 
evaluation project: 

(1) The current status of the Valley-Site A 

information system (CASA); 

(2} Site A High teachers 1 clinical perspective on 
information and features of the school's social 
organization; 

(3) Faculty-district relations as they impinge on the 
development of systemic evaluation at Site A; and 

(4) The nature of District support for the UCLA-Site A 
"reality test" and what it portends for the 
future. 

In documenting and describing each of these issues, the paper has 
been a progress report, not an account of case study findings. 
Nevertheless, it has suggested at appropriate points that the 
phenomena observed during the reality test thus far are at least 
similar to those found in other schools and school districts. The 
"tinder-utilization" of extant information is not unique to Site A, nor 
are the circumstances which appear to have led to the restricted use 
of CASA. Clinical thinking about information is sustained by the 
atomistic nature of Site A's organizational structure, but many 
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comprehensive high schools share this structure and a clinical 
perspective toward student information is widespread. Bad experiences 
with information ~ experiences in which the time-and-effort costs far 
exceed discernible benefits — are common in schools. So too are 
instances of District- faculty conflict and mistrust generated by 
contract negotiations and reductions in staff. Policy changes and 
vacillations in District support for schools 1 projects are usual, not 
extraordinary. Any "ground up" school-based information system, then, 
is likely to encounter such issues in the course of its development. 
As this paper has illustrated, such systems do not succeed or fail by 
virtue of their independent merit: on the basis of their quality of 
convenience or relevance alone. Rather, an information system and its 
social context are interdependent in dynamic, ecological balance. The 
social organizational arrangements of the school and district shape 
and sustain (or fail to sustain) the information system; and it, in 
turn, helps to shape and sustain (or fail to sustain) the arrangements 
of the school and district. It is the nature and features of this 
process which the systemic evaluation project hopes to identify and 
learn from as the reality test proceeds. 
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The Use of Survey Data 1n Comprehensive Information 
Systems for Local School Improvement: 
Purposes, Practices, and Problems 

Leigh Bursteln and Kenneth A* S1rotn1k 

The purpose of this paper 1s to examine selected Issues that 

arise 1n the use of survey data 1n comprehensive Information systems 

designed to Inform local school Improvement efforts. To achieve this 

purpose, four questions are addressed, specifically: 

1. What purpose can survey data serve 1n Information systems 
designed for local school improvements? 

2. What are the likely contents of the survey Instruments 
designed to achieve the desired purposes? 

3. What kinds of processes are necessary to develop surveys as 
elements of such systems? 

4. What are the range of routine choices that occur 1n implement- 
ing surveys as recurring components of on-going systems? 

These questions essentially deal with the why (question 1), what 
(question 2), and how (questions 3 and 4) aspects of developing 
surveys and integrating them into the information system. 

To be of value in informing practice in the use of survey data 1n 
schools, it is necessary to address the above-mentioned questions 
within context. Obviously, the purposes, contents, and processes vary 
as a function of the conditions that lead one to consider the collec- 
tion of survey data in the first place. The specific context that 
provides the impetus for this examination is an 1n-progress "reality 
test" of a school-focussed comprehensive information system being 
conducted in a suburban, California high school (to be called Site A 
throughout). This reality test has been underway since Fall 1983 and 
will continue through Fall 1985. The survey activity to date has been 
targeted predominantly on the collection of Information from students 
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with decisions about surveying other constituencies (in this context, 
teachers, parents, and the community at large) still unresolved. 

The specific context described points to the particular emphasis 
of our examination. Uses of surveys in secondary school improvement 
efforts are highlighted. The issues are those that concern school - 
based rather than district-based decision-making. The focus will 
primarily be on information collected from students than from other 
sources. Our experiences to date in the reality test will be used to 
illustrate key issues. Although these emphases are constraining to a 
certain degree, the issues that will be addressed are sufficiently 
general to be applicable to the circumstances of many schools and 
districts which are either already engaged in or are contemplating the 
use of comprehensive information systems in their improvement efforts. 

This paper is a companion document to three others that arise out 
of the same reality test. In "Contextual Influences in Developing a 
School-Based Comprehensive Information System, 11 (Dorr-Bremme, 1984), a 
brief description of the school district and school in which this 
effort is being conducted is provided along with the social 
organizational context that impinges on the development and use of the 
information system. Issues that arise in the development of reporting 
formats for communicating the system information for decision-making 
at various levels within the school are considered in "Making Sense 
Out of Comprehensive School Based Systems: Exploring Analyses and 
Reporting Methods for School Staff," (Sirotnik, 1984), and issues 
surrounding the development of the necessary computer hardware and 
software for implementing school -based information systems are 
examined in "Reporting on Distributive Processing Issues in Education 
Information Systems," (Ender, 1984). The reader should refer to these 
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documents, particularly Dorr-Bremme, (1984) for specific details about 
the educational context in which the study is being conducted. Only 
in the degree that such information is pertinent to choices in use and 
implementation of survey data will they be provided in this paper. 

In the next section, the purposes for collecting survey data to 
include in information systems for local school improvement are 
discussed. The contents of the student survey developed as part of 
the reality test are described in section 3. Other possible types of 
items that were not included, and why they weren't, are also 
indicated. The processes through which this survey was developed and 
pertinent issues in implementing and maintaining the survey are 
considered in sections 4 and 5, respectively. The paper concludes 
with projections about the issues and problems that are likely to 
arise in attempts to repeat and maintain the student survey and to 
develop surveys for other constituents to augment the existing system. 

Because the project from which the ideas about survey data use in 
school -based information systems are taken is an on-going one, the 
reader should construe this paper as a progress report rather than a 
definitive statement of findings. We anticipate providing a more 
extensive consideration of the issues as the project proceeds to 
augment th* information system through additional survey data 
collection and further implementation of the reporting system. 

The Purposes of Survey Data in Secondary School Improvement Efforts 
To begin to understand why survey data might be useful to include 
in an information sysyem f or secondary school improvement, one must 
first delineate both the intent of the improvement effort and the role 
of the information system itself in such efforts. Judging by the 
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contents of the recent commission reports on the status of education, 
by federal, state and local Initiatives to reform education, and by 
educational research on secondary schools, the concern that drives 
improvement is the perceived decline in student achievement at the 
secondary level, this decline is believed to result in a less 
qualified and prepared pool of applicants for post-secondary education 
or for smooth transition to positions in business and Industry. 

Factors most often cited as contributing to this perceived 
decline in student achievement in secondary schools are the quality of 
the curriculum and teaching and the erosion of the commitment by 
students, parents, and school personnel to high educational 
expectations and to work to achieve high levels of performance. The 
concerns about curriculum are usually expressed as beliefs that 
students take too few academic ("rigorous") courses; what they do take 
is variously described as too "narrow", "diffuse", and "low level" and 
lacks a central purpose. Questions about teaching typically point to 
inadequate training and expertise along with declining levels of 
"ability". Perceived indicators of poor commitment to educational 
achievement on the part of students, parents, and schools are the 
increases in student absenteeism (Including full -day, part-day, and 
single-period absences as well as increased tardiness), in dropouts, 
and in discipline problems (vandalism, alcohol and drug use, 
disrespectful behavior, disruptive behavior, etc.). 

Improvement efforts in secondary schools tend to be targeted on 
refocussing the school's curriculum on a coherent rigorous program 
that better prepares more students for post-secondary education or for 
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jobs. Most states are pushing this effort by Increasing graduation or 
college admission requirements (in terms of courses in core academic 
subjects), taking a closer look at teacher quality (via recruitment, 
selection, and retention procedures), and instituting an expanded 
array of achievement-based indicators by which to judge the 
performance of students and of the school they attend (as measured by 
aggregated performance). 

The external pressures for reform are clearly strong as are the 
incentives for improvement in many cases. Internal to the school 
these pressures get translated in a variety of ways. One way is for 
the school staff (administrators and teachers) to join together to 
determine what their school is about, the goals it wants to accomplish 
and the activities needed to accomplish these goals. Under the best 
of circumstances, this type of improvement effort is school-based (at 
the least, in regard to defining the activities for goal attainment), 
and school-wide (encompassing all relevant constituencies). 

In the recent initiatives for school-based educational change, 
the potential value of timely information about the school's contexts, 
processes, and outcomes to be an integral component of the improvement 
effort has received increased attention (Bank & Williams, 1981, 1983; 
Burstein, 1983a, 1984; Cooley, 1983; Hathaway, 1984; Idstein and 
Athey, 1984; Sirotnik and Oakes, 1982; Sirotnik, Burstein, & Thomas, 
1983). The advent of the technology explosion within schools has made 
the idea of accumulating a diverse set of data on students and 
programs within a computerized information system a viable and 
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potentially appealing notionl The possible contents of such systems, 
the ways in which the information system contributes to the 
improvement effort and actual examples of such systems in operation 
have been discussed in earlier reports from this project (Burstein 
1983a, 1983b, 1984; Dorr-Bremme, 1984; Sirotnik, 1984; Sirotnik et 
al., 1983; Sirotnik and Oakes, 1982) and from the CSE project on 
instructional information systems guided by Bank and Williams (1981, 
1983, 1984). 

Of most direct pertinence to the concern for the use of survey 

data in such information systems are certain premises that underly the 

work of the Systemic Evaluation Project, namely: 

0 The judicious collection, analysis, and niaintamensG of both 
achievement and non-achievement data within a comprehensive 
information system can be a key ingredient for informed 
decision-making, action taking, and evaluation that meets the 
needs at the building level (Burstein & Sirotnik, 1984b). 

0 While achievement data can serve diverse functions in schools, 
they are merely one set of symptoms of the functioning of the 
shcool culture and are best understood within that context 
through knowledge of the motivations, interests, and 
activities of the school constituencies (Burstein & Sirotnik, 
1984a) . 

The first premise points to the kinds of issues that information 
systems are intended to address. Elsewhere, we (Burstein & Sirotnik, 
1984a, 1984b) have identified three broad categories: 

(1) Examining the functioning and impact of existing school 
programs. 

(2) Monitoring key school "health" indicators such as 
course-taking patterns, absenteeism, post-graduate 
activities, participation in extra-curricular activities, 
and sentiments, attitudes, and opinions relevant to school 
held by administrators, teachers, students, parents, etc.. 
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(3) Planning, guiding, and examining new and innovative 
"in structional improvement programs. 

All these categories reflect broader, potentially school-wide 

issues. The use of information to respond to these issues informs 

decision making at a social or organizational level . As Dorr-Bremme 

points out, 

The school 1s viewed here as a cultural /ecological 
system. Its parts are conceived to be inter- 
dependent. Thus it is ideally the entire system upon 
which renewal efforts focus. This means that 
information use toward renewal 1s necessarily a 
social process... (1984, p. 20) 

Survey data are natural components of information systems 

designed to address the social, organizational issues identified 

above. As indicated in the second premise, the motivation, interests, 

and activities of school constituencies are central to decision-making 

at the system (organizational) level. Surveys provide an effective 

and efficient means of gathering this type of information. Depending 

on the operational features of the survey instrument itself and of the 

survey data collection, survey results can be used to routinely gather 

a variety of information, in a uniform manner, that can both help 

verify other information in the system (e.g., serve as an independent 

check of reported use of specific school services and programs) and 

provide "signal s 11 or messages that are harder to convey directly. An 

example of the latter might be the message provided by responses to an 

item from a teacher survey indicating that a substantial proportion of 

the teachers believe that the principal has a problem in communicating 

with the staff. The anonymity and aggregation of such a communication 

affords greater protection to teachers from overreaction by the 

principal that can occur in response to a direct (and visible) airing 

of a perceived problem. 
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Our work also pointed to a second general class of Information 
needs that a comprehensive system might fulfill. In the earlier 
phases of our reality test, the teachers we worked with did not 
routinely or easily view Information 1n terms of Us social uses. 
Instead, they were more apt to approach our questions about their 
information needs from a clinical or diagnostic perspective. That 1s, 
teachers tended to value data from the Information system that helps 
them directly 1n working with their students. 

The question that drives this clinical perspective Is whether a 
specific piece of information can contribute to refined judgements 
about a particular student or groups of students. For Instance, one 
of the teachers participating 1n our study saught data that would help 
her to form w1th1n-class heterogeneous groups to implement cooperative 
learning activities. Another teacher was Interested 1n obtaining, 
early in the semester, information that would help her refine her 
class plans to better match her class composition; while she could 
normally adjust course content based on her actual experiences with 
students, this "adjustment" process often could not be Initiated until 
the semester was well underway. A third teacher was Interested in 
data about student learning styles that would contribute to his 
decisions about what class activities to emphasize. If we had paid 
more attention to counselor uses of Information, we would have 
expected even greater emphasis on clinical applications. 

Survey data are pertinent to the clinical perspective as well as 
the social perspective. As Sirotnik (1984) documents, Items from the 
student survey contributed to both the student-level and class-level 
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reports designed for distribution to all classes early in each 
semester. Survey items on preferences in instructional activities, 
self-concept, attitudes toward school and specific subject matters 
were included in the reports as were other information (e.g., the 
number of extra- curricular activities a student is involved in and 
how many hours per week he or she is working) that could have been 
collected otherwise, but not without substantially greater costs of 
time and money. 

At a general level, then, survey data can serve two broad 
purposes in comprehensive information systems for secondary school 
improvement: 

(a) to gather information that contributes to the 
characterization of the school environment from the 
perspective of its clients in order to establish the context 
with which the reform effort must operate 

and 

(b) to assist in the diagnostic activities of school personnel 
with data gathered from school clients at or below their level 
in the educational hierarchy. 

The Contents of the Student Surveys 
As stated earlier, the general intent of collecting survey data 
is to obtain information about the characteristics, motivations, 
interests, and activities of the school constituents (students, 
teachers, administrators, parents) in order to better understand the 
functioning of the school culture. In addition, since we were 
interested, in our study, in how information aided decision-making at 
multiple levels (student, clsss, school) within a school, our 
collaborative efforts to incorporate survey data in the information 
system attempted to target topics and items usable at one or more 
level within the school. 
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The school-based work group that is participating in the devc? p- 
ment of the information system chose to focus its first year efforts 
on data about current students and programs at the high school (see 
next section for discussion of decision process) • " This descussion 
constrained the nature and contents of initial survey activities in 
several obvious ways. The time pressures exerted by the school calen- 
dar, the needs for efficient data collection and processing that could 
be managed and replicated by existing school and district personnel 
and resources, the interest in linking survey data with other 
information already available on-line through the district's computer, 
and the desire to collect information from all students across all the 
grades also affected the form and contents of the survey. 



General Description of the Student Surveys 

The final product of this first year effort, Site A Student 
Survey, May 1984 (Attachment I), was administered by third-period 
teachers to all students in attendance on Thursday, May 24, 1984. The 
contents of the survey, 185 separate questions or required responses, 
can be classified into seven major categories (see Table 1): 

1. Self -Description of Demographic Characteristics, Educational 
Expectations, and Self-concept 

2. General perceptions of Significant School Constituents 
(teacher, other students, counselors) 

3. Perception of School Climate - Issues and Problems 

4. Attitudes Toward Schooling (general and subject-matter 
related) 

5. Instruction-Related Preferences 

6. Influences on Class Selection 

7. Knowledge of and Participation in Special School Programs 
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The uneven distribution of responses across categories does not 
represent substantive emphases and concerns as much as 1t does the 
limits of the multiple choice format when, 1n some cases, a particular 
category naturally generated a checklist (e.g. specific services 
offered by the Learning Resource Center, the possible range of 
classroom materials across all types of classes) while Information 
tapped by other categories (e.g., expectations) could be collected 
more efficiently. 

A modified version of the May 1984 survey was administered to 
Incoming sophomores during a pre-school orientation program held 1n 
August 1984 (Attachment 2). This survey was intended to collect base 
Information from students about their perceptions of their high school 
prior to actual experience at the school. The survey required 104 
responses versus 185 for the Spring Survey. All questions related to 
specific experiences at the high school were either eliminated (e.g., 
items on Perceptions of Teachers and Other Students, Specific Program 
Participation, Course Selection Influences) or modified to reflect 
perceptions rather than experiences (General School Climate Issues and 
Problems) . 



Exclusion of Potentially Sensitive Questions 

There are a few features of the survey that warrant special 
mention. First, virtually all items that could be construed as 
reflecting student attitudes toward a specific administrator or 
teacher in the school were excluded. Thus students 1 attitudes toward 
the principal and their subject-matter specific instructional 
preferences were not collected. Because the student was not 
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responding anonymously (in order to be able to link survey data with 
other student Information) , 1t was felt that these type of questions 
might be too Intimidating and affect the Integrity of the responses 
throughout the survey. 

The elimination of Items directly about the principal is 
partially offset by the accumulated responses of students to questions 
about the school climate with respect to school organization and 
control. At this particular high school (like many others) the 
principal and his administration establishes the structure for 
student, teacher, and parental input to decision-making, sets the 
standards for appropriate behavior and the general tone for 
relationships among the various constituents in the school. Direct 
principal-student contacts typically occur only 1n the extremes of 
student behavior -- disciplinary action, on the one hand, and awards, 
leadership, and special services on the other. These special 
circumstances (both rpwards and sanctions) most likely influence the 
judgments of the students Involved while the rest of the student body 
relies on more indirect perceptions colored by peer, teacher, and 
parent attitudes. Under such conditions, indirect means of collecting 
Information about attitudes toward the principal may be preferred 
because it provides a potentially better balance to the biases 
introduced by the differential familiarity of students with the actual 
role of the principal. 

The exclusion of subject-matter specific instructional items 
presents a more difficult problem. Ideally, there are many 
instructional preference questions that should be targeted to 
particular content areas. The inclusion of only generic (non-course 
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specific) Items yields data that are less specifically relevant to the 
classroom decisions of a given teacher. For example, students 
expressing a preference for small group learning activities (questions 
106-109 from the spring survey) may feel differently about this 
grouping practice in their history classes than they do, say, 1n 
mathematics; the same possibility arises In attempting to interpret 
student preferences In materials and activities (questions 110-138). 
In the final analysis, the decision to ask general questions was made 
for pragmatic reasons; a longer survey would have been necessary and 
we were already concerned about length. It was also felt that 
teachers wanting more course-specific information could follow up with 
questional res of their own to further probe student preferences. 

Program Specific Questions 

A second general set of points are relevant to the sections on 
course selection and special program knowledge and participation. 
These two sections of the survey were Intended to examine specific 
ongoing activities in the school. With respect to course selection, 
the principal has recently instituted a policy that effectively 
restricted student ability to choose their teachers for particular 
courses. This policy had not been fully Implemented at the time of 
the spring survey, but the planned change had already been made known 
to students. In a sense, these items reflected different 
circumstances for different students. For seniors (who had freedom to 
choose their instructor throughout their high school years), this 
information presumably describes the basis for their typical class 
selection. But for the 10th and 11th graders, the survey represented 
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a chance to communicate their sentiments about the new "no-choice 11 
policy which they would experience. Thus the degree to which 10th and 
11th graders differed from seniors by a larger margin than they do In 
other categories of questions, reflects in part a reaction to a policy 
change. Furthermore, asking these questions In subsequent years might 
enable the administration to detect any shifts (toward greater 
acceptance of or resistance to the policy). 

The questions on the Learning Resource Center (LRC) and Career 
Magnet School (CMS) represent other Instances where the purpose of the 
data is to examine an ongoing innovation. Both programs were still 
relatively new. The questions asked, especially with respect to the 
LRC, were Intended to provide the program's director with formative 
information about how students utilized and perceived the program's 
services* In addition, survey data on both LRC and CMS were expected 
to contribute to future evaluation reports for the state-funded School 
Improvement Program which funds both programs. Thus, the planned 
survey represented an opportunity to gather Information that might 
serve external as well as internal purposes. 

Single Items Versus Scales 

Another feature of the survey content that warrants mention at 
this point is that 1n most cases, questions were included that were 
intended to stand alone as single indicators of a particular attitude, 
opinion or belief. Possibly the one major exception to this was the 
set of 14 questions on academic and social self-concept (questions 
10-23), which could operate as singletons or as scales. So far, the 
indication is that an academic self-concept scale based on questions 
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15 through 23 will be used 1n the standard report developed for use by 
teachers and counselors (See S1rotn1k & Bursteln, 1984)* 

On the one hand, using single Item Indicators 1s a parsimonious 
way to collect Information on a variety of topics; moreover, single 
Items are also easier to understand (have higher face validity) than 
multi-question scales. On the other hand, under classical notions 
about the reliability and validity of survey data, single Items have 
poorer psychometric properties than scales. 

The choice 1s not really so clear-cut when the survey data are 
Intended to Inform decision-making at various levels In a practical 
setting rather than to serve as a reliable research or diagnostic 
tool. In judging the credibility of a specific student response to a 
given question, the decision-maker must obviously consider the 
response as one piece of a larger puzzle. When considering a 
question's data aggregated over groups of students, response 
tendencies matter (the choices that attract substantial numbers of 
responses; the pattern of choices fur a question compared with those 
from related questions, e.g., an Instructional preference for having 
class discussions rather than listening to the teacher). Typically, 
these aggregate (class or school level) tendencies are more resistant 
to invalidity and unreliability than their student-level 
counterparts. One can also be somewhat reasured when the patterns of 
student responses to adjacent questions differ 1n Interpretable ways; 
such patterns usually Indicate that students are neither ignoring 
question content nor haphazardly marking items as they proceed through 
the survey. 
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Meaningful Subgroup Information 

Another concern in designing the survey was that enough 
information be gathered that could be used to form meaningful clusters 
of students that might conceivably react differently to the 
opportunities and programs operating at the school. Sex and grade are 
obvious characteristics to examine for possible differences in 
attitude. 

In our reality test, the work group was Interested 1n asking 
questions that characterized possible non-academic time commitments of 
students as well as other home factors that may affect student ability 
to focus on school work. These interests translated into questions 
about languages in the home (queston 2), the living situation 
(question 3, two parent family and other circumstances), parental help 
with school work (question 105), student employment status (question 
4), participation in extracurricular activities (questions 39-43), 
television viewing (question 5), educational aspirations and 
expectations (both parents and students, questions 6-8), and homework 
and homework effort (questions 101-102), These questions, along with 
the self-concept scales and other data about student programs of study 
contained in the information system, represent attributes that can 
potentially inform teacher and counselor diagnoses about the source of 
academic difficulties of specific students and the advisability of 
changes in the amount of outside work assigned 1n specific courses. 

At the school level, differences in student reaction to school 
policies and programs that are aligned with specific student attitudes 
can focus staff desisions on whether such differences are warranted 
and what needs to be done to eliminate them. For instance, at Site A, 
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sex differences 1n the patterns of response to questions about 
satisfaction with the counseling services offered provided empirical 
evidence that supported a request for appointment of more women to the 
counseling staff. Other survey data suggested that attitudes of 
eleventh graders are typically lower than attitudes in the other two 
grades. These results point to the need for Improvement efforts that 
employ different strategies for this grade level to bring about 
changes in their understanding of the mutual commitment of school 
staff and students to providing them appropriate educational 
opportunities. 

The above are illustrative of the ways 1n which survey data on 
student attitudes can combine with program and achievement data to 
enhance the possibilities for refined and targeted school-based 
decision-making. The reason for emphasizing the "obvious" is that the 
benefits of questions for classifying students are often short-changed 
in trying to keep questionnaire size manageable (and perhaps limit 
intruslveness) . And, except for obvious variables like sex and grade, 
school -based practitioners are not typically used to examining 
information separated by meaningful subgroups. 

Questions for Linking with Other Surveys 

In designing the student survey, we were also careful to include 
questions about school climate that might also be asked of the other 
school constituencies, particularly teachers and parents. In the 
present case, the perceptions of school function items (questions 90 
and 91) are the best example of this type of question. The 
differences that occurred in student perceptions of what the school 
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selves think (see Sirotnlk, 1984a) is useful for identifying perceived 
mis-matches in the purpose of schooling. But comparisons of student 
responses with parental and teacher attitudes are necessary to under- 
stand the depth of the conflict surrounding this Issue and to identify 
the kind of educational effort and changes that could bring 
constituent attitudes into closer agreement. Other categories of 
questions that might be asked 1n questionaires Intended for different 
constituents include items from sections on general perceptions of 
significant school constituents and general school climate issues and 
problems. 

The Survey Development Process 

The context and circumstances under which the student survey was 
developed in this project are likely to arise in secondary school 
improvement efforts in other settings (see Dorr-Bremme, 1984). The 
actual process that led to the surveys already conducted is one that 
proved "effective" and will likely be repeated in developing other 
surveys for the same school. In this section the development 
activities are briefly described. 

From the beginning of the reality test, it was clear to CSE staff 
that Site A's existing computerized information system would need to 
be augmented by the collection of student survey data. We were also 
aware that this augmentation represented an "intervention" into the 
existing Information and social organizational networks of the 
school. Our reading of the literature on implementing interventions, 
innovations and educational changes in general (e.g., Good! and, 19875; 
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Berman and McLaughlin, 1977; Bank and Williams 1981, 1983) made us 
keenly aware of the evidence that suggests that persons expected to 
directly benefit from the innovation and implement it within ongoing 
school activities should be involved in planning and designing the 
innovation. 

In building-level change efforts at the secondary level, 
mechanisms that involve school staff early on are likely to foster the 
dialogue necessary to match the innovation to the circumstances and 
interests that exist in the school and to develop a sense of 
"ownership" by the staff (and presumably the momentum and support to 
firmly plant the Innovation). In the case of the student survey, this 
involvement translates into active and intensive teacher and 
administrator input into the contents of the survey, its 
administration, and the dissemination of its results. 

The need to involve school staff in survey development has to be 
balanced against other factors that work against extensive investment 
of school staff time. Secondary school teachers lead hectic lives; 
typically, they teach five sections (usually different subject-matter) 
out of a six-period day. This leaves little time for class 
preparation and grading during school hours* On top of their class 
responsibilities, their teaching activities are regularly interrupted 
by staff development sessions, school -wide meetings, contributions to 
school and district planning and other special programs operating in 
the school, not to mention disruptions due to their personal lives 
(e.g., as parents). To give some idea of the magnitude of disruption 
on any given day, on the morning the survey results and new reporting 
formats were presented to the total staff (during a regular monthly 
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staff meeting held from 7:35 am to 9:45 am; the first three class 
periods were canceled for all students), the school still needed 
substitute coverage for 35 class periods (staff size is roughly 70), 
most of which represented teachers involved in other district business 
(e.g., curriculum planning and test development). It is expensive to 
use teacher time in developing the survey, unless the whole staff was 
committed to the effort by canceling classes, which represents an 
expense of a different sort. Substitutes are necessary to replace 
teachers wotking on system development during the day and teachers 
would have to be paid (at least in this district) to encourage them to 
meet during after-school hours. Finally, total staff involvement at 
all phases of the development would create management and timing 
problems; it simply becomes too unruly and inefficient. 

The balance we struck to obtain sufficient but non-overwhelming 
staff involvement was to form a work group of teachers and 
administrators to participate in the design of the information system, 
development of the questional' res and report formats, and planning of 
dissemination to total school staff as well as implementation into the 
school improvement effort. Our work group consists of five teachers 
representing different school departments (in our case, English, Math, 
Foreign Lanuguage, Social Studies, Health/Social Psychology), the 
coordinator of the School Improvement Program who also works as a 
counselor part-time, the vice principal responsible for prog. ams and 
the principal. Other teachers and administrators will probably be 
added during the coming years as the range of activities expand. 

The teacher participants were a mixture of volunteers who were 
already involved in many of the school's existing improvement 
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activities (including Partnership activities with UCLA, see 
Dorr-Bremme) and others who were reticent about the viability of the 
change efforts given the existing school atmosphere* The teachers 
were not necessarily expected to represent their departments but it 
was hoped that they would be sufficiently well connected with their 
colleagues to keep both other teachers and CSE project staff informed 
about the information system development and reactions to it. 

Even though the work group was kept small, it did not remove the 
need to develop early on efforts to foster mutual trust and 
understanding between school staff and CSE researchers and to limit 
time and work demands on teachers and productively use scheduled 
meetings. Visible evidence of district and school administration 
support for the project helped. It was also important to convince the 
teachers that CSE was genuinely committed to a strong role by school 
personnel in the development and use of the information system that 
could serve the school's needs rather than simply CSE's research 
agenda. (Such emphasis was consistent with project plans and 
perspectives; see Burstein and Sirotnik, 1984a, 1984b, Sirotnik et 
aK, 1983). 

The above concerns lead to a decision to begin activities of the 
work group with a series of meetings where CSE staff explained the 
general concepts that guided their interest in comprehensive 
information systems for local school improvement and provided working 
group members with details of the collaborative relationship that had 
been agreed upon by the district and school administrators (see 
Attachment 3, the topics for consideration presented at the first 
work group meeting). A summary of the purpose of the work, 
expectations substantive and operational features of the planned 
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reality test and of district and CSE commitments to the collaboration 
were provided to working group members (Burstein and Sirotnik, 1984a, 
attachment 4). This summary also helped to establish the agenda for 
the work group activities in the early phases of the project. 

After initial CSE presentations on the possible puposes of 
information systems and their use for decisions at multiple levels, 
the work group meetings concentrated on identifying the kinds of 
information the teachers, counselors, and administrators would view to 
be useful for student-level, class-lavel, school -1 evel , and 
program-specific decisions (Item 1 from the February 22 agenda, 
Attachment 3). As is characterized in other progect reports by 
Dorr-Bremme and Sirotnik, the teacher tended to focus in the early 
meetings on classroom uses of information that was primarily clinical 
in nature (see especially Dorr-Bremme, 1984, pp. 18-28) . Discussions 
surrounding social uses of information at the school level typically 
had to be initiated by CSE staff or the school administrators. 

There were four half-day work group meetings held prior to the 
May administration of the student survey. The meetings covered the 
following topics: 

1. February 22, 1984 — Description of the collaborative 
relationship and overview of project purpose; group discussion 
of information uses at different levels; teahers were asked to 
bring to the ne,tt meeting a list of the information that they 
would like to have to use at the student, class, and school 
level s. 

2. February 28, 1984 -- Discussion of information identified by 
teachers as useful; description of information already 
available in the school's computerized student system; 
discussion of the need to augment the system with survey data 
and how the survey itself could by developed. Plans made to 
distribute student survey items from Sirotnik et al . report to 
work group members who would select those items they viewed as 
appropriate for the student survey and generate other items to 
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include; CSE to compile group choices and prepare draft of 
proposed survey for next meeting. 

3. April 3, 1984 — Revision of the student survey; CSE to 
incorporate revisions into next draft. 

4. May 17, 1984 ~ Final revision of student survey; discussion 
of alternatives for administering the survey. 

Why Student Rather than Teacher Surveys 

A few comments about the topics considered and not considered at 
the meeting are warranted. First, the discussion focussed early on 
needs for information about students. Once the question of collecting 
survey data was broached, the work group members were able to identify 
a number of good reasons for devoting time to student surveying, but 
were less clear about the usefulness of surveys of teachers. This 
preference should be expected given the composition of the work group 
(predominantly teachers). Teachers could see how information about 
student attitudes, opinions, and beliefs might assist them in their 
own planning activities and in dealing with individual students; with 
some encouragement by CSE, they also recognized at least the 
possibility that survey data might be useful for school-level decision 
making. Moreover, surveys completed by students represented a limited 
intrusion on scarce teacher time. 

A teacher survey, on the other hand, was seen by some group 
members as a waste of time, both literally and figuratively* Their 
previous experiences with needs assessments conducted by the school 
and district were perceived to be of limited value to them. In some 
cases earlier surveys had been too long for the value of information 
obtained or had simply been irrelevant to teacher needs and to their 
perceptions of how changes were made in the school and district. 
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There were also occasions where teachers received no feedback of 
survey results (or at least no clearly recongnizable feedback); thus 
the time spent completing the survey was perceived to be wasted. One 
group member voiced the concern that the information could actually be 
abused because administrators at the school and district level might 
actually use it in ways detrimental to individual teachers (Sirotnik, 
1984a, discusses this situation in more detail)* In sum, efforts to 
proceed with parallel development of teacher surveys along with 
student surveys met with greater resistance (and the possibility of 
less cooperation) that might have caused the project work to bog 
down . 

In the end, a decision was made to postpone consideration of a 
teacher survey until after the student survey was developed and 
administered. In addition to the concern for its effects on the 
working relationships within the work group, practical consideration 
also supported this decision. Given the limited amount of time to 
construct and administer a student survey before the end of the school 
year, especially without unduely disrupting the work group teachers 1 
course schedules, there simply was not enough time to do both. 
Moreover, it was felt that successful development, administration, and 
reporting of the student survey might generate greater interest in 
seeking information on the attitudes of other school constituencies. 
Early reponse to the November 8, 1984 school -wide presentation of the 
survey results and other reports developed by the working groups does 
indicate teachers are now more interested in providing survey 
information about themselves for use in school improvement activities. 
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Identifying Appropriate Survey Items 

The task of developing Items for any survey can be onerous, 
especially If there are no existing surveys to serve as gudes for the 
activity. In the present case, there are lots of surveys of secondary 
school students although most have been u^ed primarily for research 
purposes. Work on climate (by Moos, Fraser, Walberg and others), 
school effectiveness (e.g., Bookover, Epstein), and school change and 
Improvement (Good! ad, Hall, Klausmeir, S1rotn1k) are good sources for 
Items that might be pertinent to the Interests at Site A. In a 
previous report (Slrotnlk et al., 1983), we had compiled Illustrative 
student survey Items, taken mainly from work on A Study of Schooling 
(Goodlad, 1983). Because of Sirotnik's previous experience with the 
functioning of these Items, we had hoped that the work group mlyht 
find some portion of them to be useful for Informing decisions at 
Site A. 

In practice, the work group relied on CSE^ survey questions to a 
greater extent than was originally envisioned. With the exception of 
the items on course selection and specific school programs, virtually 
all questions (about 130 of 185) were taksn from the CSE pool or were 
revisions of those items. 

Hindsight suggests at least three possible reasons for the heavy 
reliance on the existing survey pool. First, work group cambers were 
probably influenced by our repeated emphasis on social uses of survey 
information and, either consciously or unconsiously, yielded to our 
"authority and experience 11 by depending on our question pool for 
ideas. Second, the work group members in fact had little previous 
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experience 1n developing survey questions • Since they did not 
inherently value questionnaire development work, they were willing to 
follow the advice of persons more familiar with survey development (us 
in this case)* Presumably, if we had not been available, they would 
have used (possibly in modified form) a questionnaire developed by 
someone else, as long as it seemed relevant <o their needs. Finally, 
given competing time demands, work group members chose to be more 
reactive than proactive in the survey development once their basic 
interests were represented. Using existing questions that covered 
topics viewed as useful was a reasonable and productive decision from 
their viewpoint. 

After initial emphases were selected, the work group meetings 
honed in on whittling down a long list of survey items to a 
practically managable and useful number (less than 200 distinct 
repsonses required of students, more on this below). The main problem 
was that while work group members were concerned about devoting too 
much student time to complete the survey (the consensus was that one 
class period represented an upper time limit), each member selected 
many more questions from the available pool than could actually be 
asked and their selections did not necessarily agree; in fact almost 
all items were considered useful by at least one work group member. 

To speed up the reduction of the pool of questions, CSE staff 
compiled work group member choices and revised the survey eliminating 
questions not chosen by more than one member and also dropping 
questions that their past experience suggested wouldn't work well in 
the study setting. Another round of work group discussion helped to 
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eliminate remaining redundancies, eliminate other Items perceived as 
Interesting but perhaps not useful for the school's work, and to 
improve questionnaire layout and question wording. Even with the 
imposed constraints on length and number of question alternatives (a 
maximum of five), the final instrument was short enough for the 
machine- scorable answer sheets and the allotted time period. 
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Administration of the Survey 

Within a week of the final work group meeting, the spring survey 
was ready, printed in sufficient quantities along with pre-identified 
answer sheets, and administered to students during their third period 
classes. Out of an estimated 1678 students presumed to still be 
enrolled at the time of the survey, 1461 students (87%) completed some 
part of the student survey. The typical non-response rate for a 
question among the first 120 was around 1% with another 1 to 2% 
double-marked or out of range responses* Even at the end of the 
survey there was typically no more than 20$ non-responses. We 
interpreted these rates of usable data as evidence that the length of 
the survey was typically not a problem and that students treated the 
task of answering the survey questions seriously. 

Once the decision was made to administer a survey to incoming 
sophomores, CSE staff took a first pass at modifying the survey for 
the different population of students, ones with little direct 
knowledge of the actual functioning of the high school. The work 
group offered very few additional changes. 

The incoming sophomore survey was administered under different 
conditions than the spring survey. Students would be asked to 
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complete it during a pre-school meeting with their couselors to 
develop their course of study for the upcoming year- As a result all 
students who attended these pre-school sessions were surveyed while 
those who missed the meetings (due to being out of town) were not 
surveyed (at least for the present)- The principal estimates that 
roughly 75% of the incoming sophomores attended these sessions- The 
final count of students completing the August survey was 464- Again, 
the percentage of omits and mis-marking of answer sheets was very 
low- In fact, the main problem encountered during this survey 
administration were parents attending the session with their children 
who wanted to complete surveys of their own- 

Routing Choices in Implementing the Student Survey 
Since the reality test'work at Site A will continue into the 
coming year, in future reports, we expect to further elaborate on the 
process of developing surveys to use in the improvement effort- Here 
we briefly consider some of the routine choices that arose in the 
design and administration of the survey- They included the following: 

1. Sample of students versus entire eligible population- 

2. Survey format (Multiple choice, open-ended or mixtures) 
3- Survey length 

4. Uniform versus tailored questionnaires 

5- Identifiable versus anonymous reponses 

6- Timing of administration 

7. Actual administraton conditions 

8. Following up on absentees 
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The topics considered represent the range of mundane "nuts and bolts" 
decisions that can accur 1n Implementing surveys designed for ongoing 
school Improvement activities 1n a given school context. In our 
discussion we briefly describe the choices Involved and the decisions 
we made* 

Sample versus Census 

If the work group had been Interested only 1n survey questions to 
serve as Indicators of school-level trends 1n attitudes and beliefs, 
obtaining survey data from a sample of students would suffice 
statistically at a reduced cost 1n terms of student and teacher time. 
However, once it became clear that the work group was Interested in 
survey data to inform student level end class level decisions, there 
was no other choice than to attempt to administer the survey to all 
eligible students. 

There are drawbacks associated with surveying a sample of 
students that perhaps more than offset its advantages 1n circumbtances 
like those existing in the reality test. Ideally, a simple random 
sample would be prefered over more complex designs that, for example, 
sampled clacsrooms. But the logistics of choosing a representative 
sample of students and then arranging the administration of the survey 
to this sample can be a nighttime, and highly disruptive as well. 
Sampling classes 1s easier but less efficient given the intent to 
capture student consensus. Moreover, both sampling schemes require 
someone with a certain ^evel of technical sophistication to design, 
select, and implement the sample. Such persons are in short supply in 
most high schools. 
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Our judgment is that school staff who plan to conduct such 
surveys should choose to administer it to the entire student 
population as long as there are means for machine-tabulation of the 
responses. The extra costs of paper and student time are marginal if 
the primary concern is on quality data usable for both clinical and 
social decision-making. 

Survey Format 

The perennial aebate over the choice between selected (multiple 
choice) versus generated (open-ended) reponse is relevant to the 
present survey. Multiple choice questions are more efficient 
(assuming that machine-scorable answer sheets can be used) and more 
questions can be asked during a fixed time interval. Open-ended 
questions place fewer constraints on the repondent, thus offering the 
possibility for more diverse feedback. They also convey an aura of 
openness to free expression of opinion; this can, however, be 
intimidating for the less articulate respondent who might find it 
easier to express his or her thoughts when given a choice of possible 
alternatives. Additionally, the necessity of translating open-ended 
questionnaire responses into codable data can be a difficult and time- 
consuming task. To devote the main portion of any survey intended for 
a large sample to open-ended questions is risky, especially when 
resources are limited and quick turn-around is desirable. 

Given the desirability for timely and cost efficient handling of 
survey data intended for use in improvement efforts, the reliance on 
mainly muKiple choice questions is advisable whenever machine scoring 
of responses is possible (otherwise, the resource burdens are more 
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equal for multiple choice and open-ended formats). In our case, the 
district operates a standard test scoring system that can generate 
pre-coded and labeled answer sheets (with student names and other 
identifying characteristics), can easily tabulate responses on <a 
standard answer sheet containing 200 five choice questions, and can 
generate response frequencies and a data tape to merge with other data 
in the existing information system* Thus the multiple choice format 
was ideal, especially since it placed little additional human burden 
on the district's limited resources devoted to the reality test. We 
could have collected selected open-ended data from students in 
addition to the survey (using a different Instrument to save the 
existing questionnaire for re-use), but did not. Regardless of 
whether this was the right decision, we have not received any negative 
reactions from school staff about the absence of open-ended 
questions. Again, this may simply be a case of over reliance on our 
"expertise". 

Survey Length 

We have already mentioned the two major constraints on the length 
of a survey — the optimal amount of time required to complete it and 
the number of responses that can be coded on answer sheets in the use 
of machine-scored multiple choice questions. At the secondary level, 
a 50-minute class period represents a reasonable upper limit to impose 
on time required to provide instruction and complete an occassional 
survey. Shorter questionnaires which take less than 15 minutes to 
complete can be administered on a more frequent basis if the 
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information to be collected warrants this alternative format (e.g., 
frequent monitoring for changes in attitudes). 

Except for the constraint of a maximum of 200 responses, only 
limited consideration was devoted to the time issues by the work 
group. The teachers did feel that the final version of the spring 
survey was not too long for a single class period, which concurred 
with our own impressions based on other similar surveys. Both direct 
and indirect evidence from the actual May administration of the survey 
indicates that most students had enough time to complete the survey. 
The tendency for more non-response at the end of the questionnaire (15 
to 20% versus 1 to 2% for earlier questions) can in part be attributed 
to decisions by individual teachers (the survey was adminstered in 
class by the regular teacher after a brief training session) to devote 
only part of the class period to completing the survey (one teacher 
reportedly allowed only 10 minutes). It also may be that the 
questions at the end of the survey about the LRC and CMS were viewed 
by many students as less relevant to them, causing many to simply skip 
them. The pattern of non-reponse provides limited support for the 
latter interpretation. (Attachment 5 is a copy of the Spring survey 
containing the percentage distribution of responses.) There were no 
problems with the shorter incoming sophomore survey. On the basis of 
experience this past Spring, we will recommend keeping the student 
survey length at around 150 questions, perhaps by administering 
questions rel event to special programs on a separate occasion. 
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Uniform versus Tailored Surveys 

A decision was made to administer the same survey to all students 
during the Spring. We would have prefered to ask graduating seniors 
additional questions probing their post-graduation- plans but the 
logistics of doing this first go-around were simply too complicated. 
Besides, we felt it was unlikely that the school and district actually 
had the necessary motivation to carry out this more demanding 
administrtion and analysis task this early on. 

We didn't hesitate to shorten the survey and modify selected 
items for use with incoming sophomores. This was a clear instance 
where tailoring questions to the perspectives and knowledge of a 
specific subgroup was highly desirable. Too many questions had no 
meaning to person who had never enrolled at the Site A high school. 
Moreover, because this subgroup was administered the survey separately 
from the other students, there were no additional logistical problems 
at this phase. In terms of the computerization of these data 
(Attachment 6 reports the percentage distributions from the Incoming 
Sophomore Survey), a table of corresponding question numbers was 
sufficient to allow merger with data from the earlier survey since no 
new questions were added. 

In the future, the decisions we made might not be so clear-cut. 
There is interest in following up graduating seniors that might lead 
to special tailoring for them. If the pre-school orientation program 
(new this past year) were dropped, it might not be possible to survey 
entering 10th graders separately (though it would still be desirable 
to survey them at the beginning of their enrollment at the high 
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school). It seems to us that both the specific needs of the school 
constituencies and their sophistication at dealing with the more 
complex survey configurations associated with tailoring will determine 
how this school, and presumably others, would handle this issue 
without special assistance. 

Identifi ability Versus Anonymity 

From the outset, we had wanted to be able to identify students 
who completed a student survey. Otherwise, it would be impossible to 
integrate the survey data with the rest of the existing student 
information system. Given the uses that the work group expected to 
make of the information (particuarly for class planning and diagnostic 
purposes), it is not surprising that they supported the administration 
of student identifiable surveys. 

Despite our decision, there were, and still are, strong 
misgivings about the possibility of abuse when the responses of 
specific students can be identified. The concerns are the obvious 
ones. Teachers want to know enough to help them plan instruction but 
are worried about biasing their judgments of student needs and 
performance in the class on extra-course information (that is, 
developing inaccurate and unfair expectancies for specific students). 
Also, other students might obtain access to the responses breaching 
their confidentiality. 

In the accompanying papers by Dorr-Bremme (1984), Sirotnik 
(1984), and Sirotnik and Burstein (1984), this issue of 
confidentiality is explored at greater length. In the final analysis 
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the dilemma is a real one. Even with additional safegaurds to be 
implemented with respect to the use of the survey data in individual 
student reports, it is still necessary to place one's faith in the 
integrity and professionalism of teachers, counselors and 
administrators to use the survey data wisely. Any evidence of abuse 
of survey results would likely compromise its validity and eventually 
destroy its value for the ongoing information system. 

Timing 

The reason for administering the student survey in late Spring 
was the pragmatic one that we basically had no alternative due to time 
constraints. Ideally, there should be some more substantive basis for 
choosing when to administer the survey. Decisions about timing should 
reflect the uses intended for the data rather than the other way 
around. 

Conducting a survey near the end of the school year should yield 
better data on graduating seniors and more knowlegable experience- 
governed responses from students in their first year at the school 
(10th graders in the present case). But the student population 
typically changes by late Spring, especially in high schools. For 
those students not succeeding at the school and not strongly committed 
to the benefits of schooling as opposed to the short-term 
gratification of job earnings and limiting of possibly negative school 
experiences, dropping out late in the school year becomes an 
increasingly appealing option. So by the end of Spring, one is left 
with a more school -oriented student population. The survey simply 
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misses a segment of students (potential dropouts) that might be the 
targets of specific school reform. 

While there is no simple solution to the timing question, it does 
help to survey twice annually (early fall and late- spring) if this 
choice doesn't unduely tax school resources. Separate fall and spring 
surveys could be shorter and could include sections of common as well 
as distinct questions (e.g., more post-graduate plans and program 
information in Spring, more diagnostlcally relevant questions in the 
Fall). This choice is especially beneficial in cases where schools 
plan to place ongoing, heavy emphasis on survey data in their school 
based decision making. 

Administration 

Selecting an optimal design for administering a student survey 
during the school day can be formidable. We had originally planned to 
survey each grade level at different times during the periods special 
to them. This plan would have allowed some tailoring of the survey, 
or could have reduced costs if a common survey could be reused at the 
different grade levels. 

Instead, the survey was administered by the regular classroom 
teacher during the third-period classes. This format required more 
survey forms but simplified the packaging of pre-labeled answer 
sheets. It also required that all third period teachers be trained to 
administer the survey. As might be expected, some teachers followed 
instruction better than others; there were a few classes of studants 
who either failed to take or to complete the survey. But, at this 
point, we see no simple alternative if the survey is going to continue 
to be targeted to all students. 
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Absenteeism 

When the intent is to survey every student, one has to consider 
how to handle students who failed to complete the survey on the given 
administration date. This decision is especially .important when 
survey data are to be used as part of class-level and student-level 
decision making (as was the case here). Given the typical level of 
absenteeism (roughly 10-20$ on any given day) at the school, any 
reports with student-level data might begin to look like swiss cheese 
unless some attempt is make to follow up. 

To this point, no decision has been made about whether and how to 
gather information from students who didn't complete the survey. The 
administration of the survey was conducted on a mid-week day 
(Thursday) during a class period in the middle of the day to reduce 
the problems of "late rfsers" and "early leavers". While plans for 
follow up surveying of missing students are not firm, the present 
sense is that an attempt should be made to have counselors administer 
the survey to students missing the first round. Obviously, this 
administration would occur long after the time of the original data 
collection with more school history to contend with and perhaps a less 
cooperative set of repondents (not to mention a different 
administration procedure). These factors will have to be considered 
in greater detail before a final decision is reached for the Site A 
high school . 



91 



Page 38 



Emerging Issues: What Next? 

Other issues yet to be resolved appear throughout this report • 
There is strong sentiment to survey teachers and hopefully parents. 
Also, student survey data will continue to be collected in order to 
update student opinions for the student-level and class- level reports 
and to begin to monitor trends at the school level. The design and 
administration of these surveys will likely encounter the same sorts 
of content selection and "nuts and bolts" decisions as in the first 
student survey, probably with different solutions in certain cases 
(e.g., we expect the teacher survey data to be collected anonymously). 

During the next year we will explore the "clinical versus social 
uses of information" issues in greater detail throughout all phases of 
project work. In terms of the survey data, a question worth 
considering is how the emphasis and interests in the work group change 
when the teachers are to be the source of information. We expect less 
clinical and more social emphasis (of the "sending a message to the 
administration and district" type). We will also devote more time to 
multivariate analysis of the student survey data to explore patterns 
of differential responses to school functioning by different subgroups 
of students over time. Finally, we will monitor the reactions of 
school staff to the dissemination of the first round of survey data 
and newly designed reports. We will be looking for indicators of the 
actual ways in which the survey data are used and whether the staff 
becomes more sophisticated in their handling of the survey and other 
data as the information system has become a more visible part of the 
school's improvement efforts. 
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Table 1. Contents of the May 1984 Student Survey at Site A 



General Category 

I. Self-Description 

A. Demographic 
characteristics 



No* of 
Items 



13 



Subcategories and Specific Content' 



Sex, languages in the home, living 
situation, employment, television viewing, 
extracurricular activities, amount of 
homework, frequency of homework submission, 
parental help with work 

Parental and student educational aspirations 
student educational expectations, comfort 
with career choices 

Academic and social self-concept items 
dealing with popularity, subject-matter 
abilities, school effort and performance 



B. Expectations 



C. Self-Concept 



14 



II. General Perceptions 
of Signifcant 
School Constituents 

A, Teachers 



B. Students 



C. Counselors 



7 
7 
3 



Such descriptors as helpful, high hopes for 
students, do a good job, care about us 

Such descriptors as helpful, have high 
hopes, care about each other, competitive 



Perceptions of ease 
help with academic, 
decisions 



of getting counselor 
career, and personal 



II I .General School Climate 
Issues and Problems 

A. Students 



B. Personnel 

C, Program 



11 
6 

17 



D- School organization 
and control 6 



Misbehavior, racial conflict, drugs, 
alcohol, violence, poor attitudes, sex bias 

Teachers, counselors, administrators 

Courses, teaching, value of learning, time 
for learning 



Rules, scheduling, student and parental 
input to decision-making, selection of 
classes and teachers 



£• Physical atmosphere 3 Size, appearance, crowdedness 
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General Category 



No, of Subcategories and Specific Content 
Items 



IV. Attitudes 

A. General toward 
school 

B, Subject-matter 



C. Perceptions of 
school functions 



2 Like school, would go to different school 



9 Math, English, arts, social studies, 

science, foreign languages, business and 
economics, physical education 



Student perceptions of school and own 
beliefs about most important function of 
schooling 



V • Instruc ti on-Rel ated 

Atti tudes 33 

VI. Influences on class 

choice 14 



VI I. Special Program 

and Knowledge 34 



Preferences in instructional grouping, 
activities, materials and characteristics 

Student perceptions of factors affecting 
choice of classes 



Specific items on two special programs 
(Career Magnet School and Learning Resource 
Center) 
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Attachment 1 
High School Student Survey 

May 24, 1984 

The survey you are about to complete will ask you questions about 
yourself and about your school. This is not a test. There are no right or 
wrong answers. The survey will give you an opportunity to express how you 
feel about what happens in your classes and around school. That is why it 
is important to answer the questions as truthfully and as carefully as 
possible. 



DO NOT WRITE ON THESE PAGES 

MARK YOUR ANSWERS ON THE ANSWER SHEET PROVIDED. You will notice that 
answers go from A to E or from F to K. This .does not matter. Simply 
choose the ^ne answer that best fits your opinion for each question. MARK 
ONLY ONE LETTER ON THE ANSWER SHEET FOR EACH QUESTION. For example, if you 
chose answer B for question number 5, you would mark the answer sheet like 
this: 

A B C O E 

5O0OOO 



Or, if you chose answer J for question number 6, you would mark the answer 
sheet like this: 

FGHJK 

iOOO«0 

Remember, mark only one letter on the answer sheet for each question. If 
there are any words or questions you don't understand, please raise your 
hand and ask for help. 



DO NOT BEGIN UNTIL YOU RECEIYE MORE INSTRUCTIONS 



This question will be answered differently than the others. You will 
use the blue box at the top of the answer sheet. Read the list of Career 
Magnet Schools below. 



1- Physical Science and Technology 

2. International Relations S Political Science 

3. Business 

4. Industry 

5o Performing, Visual and Fine Arts 

6. Mental, Physical A Biological Sciences 

7. Liberal Arts 

8. Entry and Essentials 

9. Don't Know 



Now, using the last column of the blue box (to the far right), mark the 
number on the answer sheet that matches your career magnet school . 
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Starting with number 1 on the survey, the rest of the questions will 
be answered in the white area of the answer sheet. Remember, do not mark 
on the survey sheets themselves. Mark one answer for each question on the 
answer sheet. 
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Questions About Yourself 



1. Sex: 

A. Male 

B. Female 

2. Besides English, what other languages are spoken in your home: 

F. None 

6. Spanish 

H. Vietnamese 

J. Chinese 

K. Other 

3. Living situation: 

A. With two parents (Includes stepparents) 

B. With one parent only (mother or father only) 

C. Guardian(s) /foster parents 

D. Alone or with friends 

E. Other 

4. About how many hours a week do you usually spend working on a job during the school year? 

F. None. I am not employed during the school year. 

G. About 10 hours or less 

H. About 15 - 20 hours 
J. About 20 - 30 hours 
K. More than 30 hours 

5. How many hours do you w?*ch television each day? 

A. None 

B. About 1 hour 

C. About 2-3 hours 

D. About 4-5 hours 

E. More than 5 hours 

Choose the ONE answer thut best completes each of the following sentences. 

6. If I could do anything I want, I would like to: 

F. Quit school as soon as possible. 

G. Finish high school. 

H. Go to trade/technical school or junior college. 
J. Go to a 4-year college or university. 

K. Don't know. 

7. I think ny parents would like me to : 

A. Quit school as soon as possible. 

B. Finish high school. 

C. Go to trade/technical school or junior college. 

D. Go to a 4-year college or university. 

E. Don't know. 
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8. Actually, I will probably : 

F. Quit school as soon as possible. 

G. Finish high school. 

H. Go to trade/technical school or junior college. 
J. Go to a 4-year college or university. 

K. Don't know. 

9. Hew comfortable do you feel about choosing a future career goal at this point in your life? 

A. Very Uncomfortable 

B. Uncomfortable 

C. Neither Uncomfortable or Comfortable 

D. Comfortable 

E. Very Comfortable 

The following sentences describe some of the ways in which people might think about themselves* 

Read each of the following sentences carefully and mark the letter on the answer sheet that 
tells how much it 1s like you. 

Look at the following practice sentence and mark the letter on the answer sheet that tells how 
much you agree or disagree with the sentence. 



PRACTICE 


Strongly 


Mildly 


Not 


Mildly 


Strongly 




Agree 


Agree 


Sure 


Disagree 


Disagree 


I am good at art 


A. 


B. 


C. 


D. 


E. * 



If you Choose "Strongly Agree," you're saying that you are very good at art. If you choose 
"Mildly Agree," you're saying that you are OK at art. If you choose "Mildly Disagree," you're - 
saying that you are not too good at art. If you choose "Strongly Disagree," you're saying that 
you are very poor at art. 

Strongly Mildly Mot Mildly Strongly 







Agree 


Agree 


Sure 


Disagree 


Disa 


10. 


I'm popular with kids my own age. 


F. 


G. 


H. 


J. 


K. 


11. 


Kids usually follow my ideas. 


A. 


B. 


C. 


D. 


E. 


12. 


Most people are better liked than I am. 


F. 


G. 


H. 


J 


K. 


13. 


It is hard for me to make friends. 


A. 


B. 


C. 


D. 


E. 


14. 


I have no real friends. 


F. 


G. 


H. 


J. 


K. 


15. 


I'm not doing as well as I'd like to in school. 


A. 


B. 


C. 


D. 


E. 


16. 


I am a good reader. 


F. 


G. 


H, 


J. 


K. 


17. 


I'm proud of my school work. 


A. 


B. 


C. 


D. 


E. 


18. 


I'm good at math. 


F. 


G. 


H. 


J. 


K. 


19. 


I'm doing the best work that I can. 


A. 


B. 


C 


D. 


E. 


20. 


I am able to do school work at least as well as 
other students. 


F. 


G. 


H. 


J. 


K. 
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Strongly 


Mildly 


Not 


Mildly 


Strongly 








/vy ret? 


otire 


ui Sayree 


ui sagree 


21. 


My grades are not good enough. 


A. 


B. 


c. 


D. 


E. 


22* 


I'm always making mistakes in ny schoolwork. 


F. 


6. 


H. 


J. 


K. 


23. 


I am a good writer. 


A. 


B. 


C. 


D. 


E. 



Questions About Your School Life 
How much do the following words describe most of the teachers at this school? 





Very 


Pretty 


Some- 


Only A 


Not at 






rajcn 


what 


Little Bit 


Ml 1 

All 


24. Friendly 


F. 


G. 


H. 


J. 


K. 


25. Helpful 


A. 


B. 


C. 


D. 


E. 


26* Have high hopes for us 


F. 


6. 


H. 


J. 


K. 


27. Talk to us 


A. 


B. 


C. 


D. 


E. 


28. Let us talk to then 


F. 


G. 


H. 


J. 


K. 


29. Care about us 


A. 


B. 


C. 


D. 


E. 


30. Do a good job 


F. 


G. 


H. 


J. 


K. 


How much do the following words describe 


how you feel 


about most 


* 

of the students at this 




school? 














Very 


Pretty 


Some- 


Only A 


Not at 




Much 


Much 


what 


Little Bit 


All 


31. Friendly 


A. 


B. 


C. 


D. 


E. 


32. Helpful 


F. 


G. 


H. 


J. 


K. 


33, Have high hopes 


A. 


B. 


C. 


D. 


E. 


34. Smart 


T? 

r • 


n 

VI. 


u 
ii. 


.i 


K, 


35. Talk to each other 


A. 


B. 


c. 


D. 


E. 


36. Care about each other 


F. 


G. 


H. 


J. 


K. 


37. Competitive 


A. 


B. 


C. 


D. 


E. 
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38. The most popular students in this school are: (Choose only one arswer) 

F. Athletes 

6. Smart students 

H. Meiribers of student government 

J. Good-looking students 

K. Wealthy students 

Indicate whether or not you participate in the following activities at school. (Answer yes or 
no for each of the following). 

Yes No 

39. I participate in sports teams/drill team/flags/cheerleading. A. B. 

40. I participate in student government. F* G. 

41. I participate in music, band, drama, or other arts. A. B. 

42. I participate in honor society. F. G. 

43. I participate in school c\ Jbs/ccxmunity service activities. A. B. 

Below is a list of things which may be problems at this school. Hew much do you think each is a 
problem at this school? 

Not a Minor Major 







Problem 


Problem 


Probl em" 


44. 


Student misbehavior (fighting, stealing, gangs, truancy, etc.) 


F. 


G. 


H. 


45. 


Poor courses or not enough different subjects offered 


A. 


B. 


C. * 


46. 


Prejudice/Racial conflict 


F. 


G. 


H. 


47. 


Drugs 


A. 


B. 


C. 


48. 


Alcohol 


F. 


G. 


H. 


49. 


Poor teachers or teaching 


A. 


B. 


C. 


50. 


School too large/classes overcrowded 


F. 


G. 


H. 


51. 


Teachers don't discipline students. 


A. 


B. 


C. 


52. 


Poor or not enough buildings, equipment, or materials 


F. 


G. 


H. 


53. 


The principal and other people in the office who run the school 


A. 


B. 


C. 


54. 


Poor student attitudes (poor school spirit, don't want to learn) 


F. 


G. 


H. 


55. 


Too many rules and regulations 


A. 


B. 


C. 


56. 


How the school is organized (class schedules, not enough time 










for lunch, passing periods, etc.) 


F. 


G. 


H. 
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Issues and Problems : 

Read each one of the following sentences carefully and choose the letter that tells how much you 
agree or disagree with what it says. CHOOSE ONLY CHE LETTER for each sentence. Please raise 
your hand if you have aoy questions. 



Strongly 
Agree 



57. What I'm learning in school is useful 
for what I will need to know NOW. 

58. What I'm learning in school will be 
useful for what I will need to know 
LATER in life. 

59. I think students of different races or 
colors should go to school together. 

60. Girls get a better education than boys 
at this school . 

61. There are places at this school where 

I don't go because I'm afraid of other 
students. 

6?. Boys get a better education than girls 
at this school. 

63. I do not have enough time to do my school 
work. 

64. High school students should have job 
experience as part of their school 
program. 

65. Many students at this school don't 
care about learning. 

66. Average students don't get enough 
attention at this school. 

67. Some of the things teachers want me to learn 
are just too hard. 

68. Too many students are allowed to 
graduate from this school without 
learning very much. 

69. If I had my choice, I would go to a 
different school. 

70. There are things I want to learn 
about that this school doesn't 
teach. 

71. It's not safe to walk to and from 
school alone. 



A. 

F. 
A. 
F. 

A. 
F. 
A. 

F. 
A. 
F. 
A. 

F. 

A. 

F. 

A. 



Mildly 
Agree 



B. 

G. 
B. 
G. 

B. 
G. 
B. 

G. 
B. 
G. 
B. 

G. 
B. 

G. 
B. 



Not 
Sure 



C. 

H. 
C. 
H. 

C. 
H. 
C. 

H. 
C. 
H. 
C. 

H. 
C. 

H. 
C. 



Mildly 
Disajjc-ee 



D. 

J. 
0. 
J. 

D. 
J. 
D. 

J. 
D. 
J. 
D. 

J. 
D. 

J. 
D. 



Strongly 
Disagree 



E. 

K. 
E. 
K. 

E. 
K. 
E. 

K. 
E. 
K. 
E. 

K. 
E. 

K. 
E. 
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Strongly Mildly Not Mildly Strongly' 
Agree Agree Sure Disagree Disagree 



72. I have trouble reading the books and other 



materials in my classes* 


F. 


G. 


H. 


J. 


K. 


73. Thp arades or mark*; I apt hpln mp tn Iparn 
better. 


A. 


■ B. 


C. 


D. 


E. 


74. I like school. 


F 


e 

Q« 


u 
ru 


u • 




75. The grades or marks I get in class have 
nothina to do with what I reallv know. 


A 


D 
D* 


r 


n 
u. 


r 
L. 


76. I have to learn things without knowing why. 


F. 


G. 


H. 


J. 


K. 


77. Parents should have a say in what is 
taught at this school. 


A. 


B. 


C. 


D. 


E. 


78. It is easy for me to get help from a 
counselor when planning ny school program. 


F. 


G. 


H. 


J. 


K. 


79. We are not given enough freedom in choosing 
our classes. 


A. 


B. 


C. 


D. 


E. 


80. We are not given enough freedom in 
choosing our teachers. 


F. 


G. 


H. 


J. 


K. 


81. If I have a personal problem, it would be 
easy for me to get help from a counselor. 


A. 


B. 


. c. 


D. 


E. 


82. If you don't want to go to college, this 
school doesn t think you re very important. 


F. 


G. 


H. 


J. 


K. 


83. Students should have a say in what is 
taught at this school. 


A. 


B. 


C. 


D. 


E. 


84. A person is foolish to keep going to 
school if he/she can get a job. 


F. 


G. 


H. 


J. 


K. 


t*\ am ▼ ^ ▼ ft 1 ^ ^ • * 

85. If I need help planning for a career, it 
would be easy for me to get help from a 

i^nnnQPl n** 

UUUI IjC 1 Ul • 


A 

A. 


Q 

D. 


n 
L. 


D. 


E. 


86. I like the way this school looks. 


F. 


G. 


H. 


J. 


K. 


87. It is easy to get books from the 
school library. 


A. 


B. 


C. 


D. 


E. 


88. Things in the school librae are useful 
to me. 


F. 


G. 


H. 


J. 


K. 


89. Materials in the Career Guidance Center (CGC) 
are useful to me. 


A. 


B. 


C. 


D. 


E. 



DO NOT WRITE ON THIS PAGE 



9 

ERIC 



-6- 

103 



Questions About Teaching, Learning & Classroom Work 

All schools teach pretty much the same things, but they nqy think some things are 
more inportant than others. . . 

90. Which ONE of these does this school think is the most important thing for students? (Choose 
only one ) 

F. To work well with other people 

G. To learn the basic skills in reading, writing, arithmetic, and other subjects 

H. To become a better person 
J. To get a good job 

91. If you had to choose only the ONE most important thing for you , which would it be? (Choose 
only one) 

A. To work well with other people 

B. To learn the basic skills in reading, writing, arithmetic, and other subjects 

C. To become a better person 

D. To get a good job 

In general, how do you like the following subjects? 







Like 
Very 
Much 


Like 
Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


92. 


English 


F. 


G. 


H. 


J. * 


K. 


93. 


Mathematics 


A. 


B. 


C. 


D. 


E. 


94. 


Social studies (history, geography, 
government, etc.) 


F. 


6. 


H. 


J. 


K. 


95. 


Science 


A. 


B. 


C. 


D. 


E. 


96. 


Computer Education 


F. 


G. 


H. 


J. 


J. 


97. 


The Arts (art, crafts, music, drama, 
dance, creative writing, film- 
making, photography) 


A. 


B. 


C. 


D. 


E. 


98. 


Foreign Language 


F. 


G. 


H. 


J. 


K. 


99. 


Vocational /Career Education (shop, 
business education, home economics, 
etc.) 


A. 


B. 


C 


D. 


E. 


100. 


Physical Education 


F. 


G. 


H. 


J. 


K. 
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101. How many hours of homework do you have each day? 

A. None 

B. About 1 hour 

C. About 2-3 hours 

D. About 4-5 hours 

E. More than 5 hours 

102. In general, how often do you do your homework? 

F. All of the time 

G. Most of the time 

H. Sometimes 
J. Seldom 

K. Never 

103. How soon do teachers usually return your work? 

A. the next day 

B. 2 days later 

C. 3 days later 

D. 4 days later 

E. 5 days later or more 

104. When you make mistakes in your work, how often do teachers tell you how to do it correctly? 

F. All of the time 

G. Most of the time 

H. Only sometimes 
J. Seldom 

K. Never 

105. How often do your parents or other family rrenbers help you with your school work? 

A. All of the time 

B. Most of the time 

C. Only sometimes 

D. Seldom 

E. Never 

Listed be^ow are four ways students can work in a classroom. Choose the letter on the answer 
sheet that tells how much you liEe or would like to work in each way, even if you don't do so 
now. 







Like 
Very 
Much 


Like 

Scoewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


106. 


Alone by myself 


F. 


G. 


H. 


J. 


K. 


107. 


With the whole class 


A. 


B. 


C. 


D. 


E. 


108. 


With a small group of students, 
who knew as much as 1 do 


F. 


G. 


H. 


J. 


K. 


109. 


With a small group of students, 
some who know less, some who know 
as much, anc some who know more 
than I do 


A. 


B. 


C. 


D. 


E. 
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Listed belcw are some things that night be used in a class . Choose the letter on the answer 
sheet that tells how much you like or would like to use each thing, even if you don't use it in 
a classroom. 





Like 
Very 
Much 


Like 

Sor/swhat 


. undecided 


Dislike 
Somewhat 


Dislilcp 

Ml Jl i KC 

Very Much 


110. Textbooks 


F. 


G. 


H. 


J» 


K. 


111. Other books 


A. 


B. 


C. 


D. 


E. 

u* 


112- Work sheets 


F. 


Q. 


H. 


J. 


rv. 


113. Filrcs, fllmstrips, cr slides 


A. 


B. 


c. 


n 

far. 


r 
c. 


114* Gaines or simulations 


• • 


G. 


H. 


J 
U . 


rs* 


115. Newspapers or magazines 


A. 


B. 


C. 


n 


c 

c 


116. Tape recordings or records 


F. 


6. 


H. 


0. 


K. 


117. Television/video 


A. 


B. 


G. 


D. 


E. 


118. Calculators 


F. 


u. 


H. 


0. 


K. 


119. Globes, maps, and charts 


A. 


B. 


C. 


D. 


E. 


120. Aninals and plants 


F. 


6. 


H. 


J. 


K. 


121. Lab equipment and materials 


A. 


B. 


C. 


D. 


E. 


122. Conputers 


F. 


G. 


H. 


J. 


K. 


Listed belorf are some things that you might do in a class, 
sheet that tells how much you like or would like to do each 
class. 


Choose the letter on the answer 
thing, even if you don't do it in 




Like 
Very 
Much 


Like 

Sorewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


123. Listen to the teacher 


A. 


B. 


C. 


D. 


E. 


124. Go on field trips 


F. 


G. 


H. 


J. 


K. 


125. Do research and write reports, 
stories, or poems 


A. 


B. 


C. 


D. 


E. 


126. Listen to student reports 


F. 


G. 


H. 


0. 


K. 


127. Lister, to speakers who come to class 


A. 


a 


C. 


n. 


F. 


128. Have class discussions 


F. 


G. 


H. 


J. 


K. 


129. Build or draw things 


A. 


B. 


C. 


D. 


E. 


130. Do problems or write answers to 
questions 


F. 


G. 


H. 


J. 


K. 
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Like 
Very 

Much 


Like 

Somewhat Undecided 


Dislike 


Dislike 
very men 


131 ♦ Take tests or quizzes A* 


B. 




C. 




D. 


E. 


132. Make films or recordings F. 


u. 




U 




J. 


K. 


133. Act things out A. 


B. 


• 


c. 




D. 


E. 


134. Read for fun or interest F. 


G. 




H. 




J. 


K. 


135. Read for information A. 


B. 




C. 




D. 


E. 


136. Interview people F. 


G. 




H. 




J. 


K. 


137. Dc projects or experiments that 

are already planned A. 


B. 




C. 




0. 


E. 


138. Do projects or expariments that I 

plan F. 


G. 




H. 




J. 


X. 


Please indicate how important each of the following items was in your choice 
of classes here at High School, 




Very 
Important 


Important 


NOt 

Sure 


NOt 

Important 


Very 

umroporcant 


139. Taking classes from teachers I like 


A. 


B. 




C. 


D. 


E. 


140. Being in the same classes as my friends 


F. 


G. 




H. 


J. 


K. 


141. Completing graduation requirements 


A. 


B. 




C. 


D. 


E. 


142. Learning skills for a future job 


F. 


G. 




H. 


J. 


K. 


143. Taking classes that will help to be a better 
person 


A. 


Bo 




C. 


D. 


E. 


144. Being challenged by taking hard subjects 


F. 


G. 




H. 


J. 


K. 


145. Taking classes that will prepare me for the 
future 


A. 


B. 




C. 


D. 


E. 


146. Getting a wide variety of classes 


F. 


G. 




H. 


J. 


K. 


147. Preparing for college 


A. 


B. 




C. 


D. 


E. 


148. Taking classes requiring little work 


F. 


G. 




H. 


J. 


K. 


149. Avoiding subjects I don't like 


A. 


B. 




C. 


D. 


E. 


150. Taking classes that are popular 


F. 


G. 




H. 


J. 


K. 


151. Taking classes ny parent(s) consider 














important 


A. 


B. 




C. 


D. 


E. 


152. Taking classes where I can get good grades 


F. 


G. 




H. 


J. 


K. 
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Questions About the teaming Resource Center (LRC) 

153. Have you heard of the Learning Resource Center? 

A. yes 

B. no 

154. If yes, hew often have you gone with your classes to the Learning Resource Center? 

F. Never 

G. Only once or twice 

H. About once or twice a month 
J. About once or twice a week 
K. Almost every dey 

1 55. How often have you gone to the Learning Resource Center by yourself? 

A. Never 

B. Only once or twice 

C. About once or twice a month 

D. About once or twice a week 

E. Almost every day 



If you have ever used the Learning Resource Center, have you used ar\y of these services? 
(Answer yes or no for each of the following). 







Yes 


No 


156. 


Diagnostic testing for reading and math problems 


F. 


G. 


157. 


Entry testing for proper class placement 


A. 


B. 


158. 


Assistance with assignments from classroom teacher 


F. 


G. 


159. 


Work on tasks assigned by the Learning Resource Center 


A. 


B. 


160. 


After school seminars 


F. 


G. 


161. 


Study hall 


A. 


B. 


162. 


SAT preparation 


F. 


G. 


163. 


Proficiency test preparation 


A. 


B. 


164. 


Use the computer 


F. 


G. 


165. 


Sophomore study skills 


A. 


B. 


166. 


Language laboratory 


F. 


G. 


167. 


Assistance 1n researching or typing papers 


A. 


B. 


168. 


Use the typewriter 


F. 


G. 


169. 


Receive Individual tutoring 


A. 


B. 


nn 


Develop library/research skills 


F. 


G. 


171. 


Develop reading skills 


A. 


B. 


172. 


Develop writing skills 


F. 


G. 


173. 


Develop math skills 


A. 


B. 


174. 


Develop listening skills 


F. 


G. 


175. 


Develop test taking skills 


A. 


B. 
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176. Have you received credit for Writing I through the Learning Resource Center? 

F, yes 
6. no 

177. Have you received credit for Devel oprnental Reading through the Learning Resource Center? 

A. yes 

B. no 



Strongly 
Agree 



178. The Learning Resource Center is 

helping students at Royal. F. 

179. Most students know about the resources 
available in the Learning Resource 
Center, A. 

180. I have been helped by the services of 

the Learning Resource Center. F. 

181. I am comfortable about using the services 
of the Learning Resource Center. A. 

182. fty work in the Learning Resource Center 
has helped me in ny courses. F. 

183. work in the Learning Resource Center 
has made me feel more secure about ny 
ability to do the work assigned by 
teachers. A. 



Agree 
G. 

B. 
G. 
B. 
G. 

B. 



Undecided 
H. 

C. 
H. 
C. 
H. 

C. 



Strongly 
Disagree Disagree 



J. 

D. 
J. 
D. 

J. 

D. 



K. 

E. 
K. 
E. 
K. 

E. 



Questions About the Career Magnet School 
Strongly 



Agree 

184. I understand what the Career Magnet 
School program is trying to do. F. 

185. I would like more infonnation about the 
Career Magnet Schools. A. 



Agree Undecided 



G. 
B. 



H. 



C. 



Strongly 
Disagree Disagree 



J. 
D. 



K. 
E. 
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High School Student Survey 
Incoming Sophomores 

August 22-23, 1984 

The survey you are about to complete will ask you questions about 
yourself and about your experiences at school. This is not a test. There 
are no right or wrong answers. It is important to answer the questions as 
truthfully and as carefully as possible. 



DO HOT WRITE ON THESE PAGES 

MARK YOUR ANSWERS ON THE ANSWER SHEET PROVIDED. You will notice that 
answers go from A to E or from F to K. This does not matter. Simply 
choose the one answer that best fits your opinion for each question. MARK 
ONLY ONE LETTER ON THE ANSWER SHEET FOR EACH QUESTION. For example, if you 
chose answer B for question number 5, you would mark the answer sheet like 
thi s : 



Or, if you chose answer J for question number 6, you would mark the answer 
sheet like this: 



Remember, mark only one letter on the answer sheet for each question. If 
there are any words or questions you don't understand, please raise your 
hand and ask for help. 



DO NOT BEGIN UNTIL YOU RECEIVE MORE INSTRUCTIONS 

This question will be answered differently than the others. You will 
use the blue box at the top of the answer sheet. Read the list of Career 
Magnet Schools below. 

1. Physical Science and Technology 

2. International Relations & Political Science 

3. Business 

4. Industry 

5. Performing, Visual and Fine Arts 

6. Mental, Physical & Biological Sciences 

7. Liberal Arts 

8. Entry and Essentials 

9. Don't Know 

Mow, using the last column of the blue box (to. the far right), mark the 
number on the answer sheet that matches your career magnet school . 

Starting with number 1 on the survey, the rest of the questions will 
be answered in the white area of the answer sheet. Remember, do not mark 
on the survey sheets themselves. Mark one answer for each question on the 
answer sheet. 
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Questions About Yourself 

1. Sex: 

A. Male 

B. Female 

2. Besides English, what other languages are spoken in your home: 

F. None 

6. Spanish 

H. Vietnamese 

J. Chinese 

K. Other 

3. Living situation: 

A. With two parents (includes stepparents) 

B. With one parent only (mother or father only) 

C. Guardian(s) /foster parents 

D. Alone or with friends 

E. Other 

4. About how many hours a week do you usually spend working on a job during the school year? 

F. None. I am not employed during the school year. 

G. About 10 hours or less 

H. About 15 - 20 hours 
J. About 20 - 30 hours 
K. More than 30 hours 

5. How maty hours do you watch television each day? 

A. None 

B. About 1 hour 

C. About 2-3 hours 

D. About 4-5 hours 

E. More than 5 hours 

Choose the ONE answer that best completes each of the following sentences. 

6. If I could do anything I want, I would like to: 

F. Quit school as soon as possible. 

G. Finish high school. 

K. Go to trade/technical school or junior college. 

J. Go to a 4-year college or university. 

K. Don't know. 

7. I think ny parents would like me to : 

A. Quit school as soon as possible. 

B. Finish high school. 

C. Go to trade/ technical school or junior college. 

D. Go to a 4-year college or university. 

E. Don't know. 
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8. Actually, I will probably : 

F. Quit school as soon as possible. 

G. Finish high school. 

H. Go to trade/technical school or junior college. 
J. Go to a 4-year college or university. 

K. Don't know. 

9. How comfortable do you feel about choosing a future career goal at this point in your life? 

A. Very Uncomfortable 

B. Uncomfortable 

C. Neither Uncomfortable or Comfortable 

D. Comfortable 

E. Very Comfortable 

The following sentences describe some of the wsys in vtoich people might think about themselves. 

Read each of the following sentences carefully and mark the letter on the answer sheet that 
tells how much it is l ike y ou. 

Look at the following practice sentence and mark the letter on the answer sheet that tells how 
much you agree or disagree with the sentence. 

PRACTICE Strongly Mildly Not Mildly Strongly 

Agree Agree Sure Disagree Disagree 

I am good at art * A. B. C. D. E. 

If you Choose "Strongly Agree," you're saying that you are very good at art. If you choose 
"Mildly Agree," you're spying that you are OK at art. If you choose "Mildly Disagree," you 1 re 
saying that you are not too good at art. If you choose "Strongly Disagree," you're saying that 
you are very poor at art. 







Strongly 
Agree 


Mildly 
Agree 


Not 
Sure 


Mildly 
Disagree 


Stroni 
Di sat 


10. 


I'm popular with kids my own age. 


F. 


G. 


H. 


J. 


K. 


11. 


Kids usually follow my ideas. 


A. 


B. 


C. 


D. 


E. 


12. 


Most people are better liked than I am. 


F. 


G. 


H. 


J 


K. 


13. 


It is hard for me to make friends. 


A. 


B. 


C. 


D. 


E. 


14. 


I have no real friends. 


F. 


G. 


H. 


J. 


K. 


15. 


I'm not doing as well as I'd like to in school 


. A. 


B. 


C. 


D. 


E. 


16. 


I am a good reader. 


F. 


G. 


H. 


J. 


K. 


17. 


I'm proud of my schoolwork. 


A. 


B. 


C. 


D. 


E. 


18. 


I'm good at math. 


F. 


G. 


H. 


J. 


K. 


19. 


I'm doing the best work that I can. 


A. 


B. 


C. 


D. 


E. 


20. 


I am able to do schoolwork at least as well as 
other students. 


F. 


G. 


H. 


J. 


K. 
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Strongly 


Mildly 


Not 

OUI c 


Mildly 


Strongly 


21. 


My grades are not good enough. 


A. 


B. 


c. 


D. 


E. 


22. 


I'm always making mistakes in ny schoolwork. 


F. 


G. 


H. 


J. 


K. 


23. 


I am a good vHter. 


A. 


B. 


C. 


D. 


E. 



Questions About Your School Life . 

24. Who do you think the most popular students in this school are? (Choose only one answer) 

F. Athletes 

G. Staart students 

H. Members of student government 
J. Good-looking students 

K. Wealthy students 

Below 's a list of things which may be problems at this school. How much do you think each is a 
problem at this school? 







Not a 
Problem 


Minor 
Problem 


Major 
Problem 


25. 


Student misbehavior (fighting, stealing, gangs, truancy, etc.) 


A. 


B. 


C. 


26. 


Poor courses or not enough different subjects offered 


F. 


G. 


H. 


27. 


Prejudice/Racial conflict 


A. 


B. 


C. 


28. 


Drugs 


F. 


G. 


H. 


29. 


Alcohol 


A. 


B. 


'c. 


30. 


Poor teachers or teaching 


F. 


G. 


H. 


31. 


School too large/classes overcrowded 


A. 


B. 


C. 


32. 


Teachers don't discipline students. 


F. 


G. 


H. 


33. 


Poor or not enough buildings, equipment, or materials 


A. 


B. 


C. 


34. 


The principal and other people in the office who run the school 


F. 


G. 


H. 


35. 


Poor student attitudes (poor school spirit, don't want to learn) 


A. 


B. 


C. 


36. 


Too many rules and regulations 


F. 


G. 


H. 


37. 


How the school is organized (class schedules, not enough time 
for lunch, passing periods, etc.) 


A. 


B. 


C. 
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Issues and Problems: 



Read each one of the following sentences carefully and choose the letter that tells how much you 
agree or disagree with what it says. CHOOSE ONLY ONE LETTER for each sentence. Please raise 
your hand if you have any questions. 



38. 
39. 

40. 
41. 
42. 

43. 

44. 

45. 

46. 

47. 

48. 
49. 

50. 
51. 

52. 

53. 

54. 

55. 





Strongly 
Agree 


Mildly 
Agree 


Not 
Sure 


Mildly 
Di sagree 


Strongly 
Disagree 


What I'm learning in school is useful 
for what I will need to know NOW. 


F. 


G. 


H, 


J. 


K. 


What I'm learning in school will be 
useful for vtfiat I will need to know 
LATER in life. 


A. 


B. 


C. 


D. 


E. 


I think students of different races or 
colors should go to school together. 


F. 


G. 


H. 


J. 


K. 


I do not have enough time to do ny school 
work. 


A. 


B. 


C. 


0. 


E. 


High school students should have job 
experience as part of their school 
program. 


F. 


G. 


ii 

H. 


1 

J. 


U 

N. 


Some of the things teachers want ms to learn 
are just too hard. A. 


n 
D. 




n 


P 
r - » 


If I had ny choice, I would go to a 
different school . 


F. 


G. 


H. 


j. 


K. 


It's not safe to walk to and from 
school alone. 


A. 


B. 


C. 


0. 


E. 


I have trouble reading the books and other 
materials in ny classes. 


F. 


G. 


H. 


j. 


K. 


The grades or marks I get help me to learn 
better. 


A. 


B. 


C. 


D. 


E. 


I like school. 


F. 


G. 


H. 


J. 


K. 


The grades or marks I get in cl ass have 
nothing to do with what I really know. 


A. 


R 
u • 


r 


n 


F. 


I have to learn things without knowing why. 


F. 


G. 


H. 


J. 


K. 


Parents should have a say in what is 
taught at this school. 


F. 


G. 


H. 


J. 


K. 


We are not given enough freedom in choosing 
our classes. 


A. 


B. 


C. 


0. 


E. 


Students should have a say in what is 
taught at this school. 


F. 


G. 


H. 


J. 


K. 


A person is foolish to keep going to 
school if he/she can get a job. 


A. 


B. 


C. 


0. 


E. 


I like the way this school looks. 


F. 


G. 


H. 


J. 


K. 
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Questions About Teaching, Learning & Classroom Work 



All schools teach pretty much the same things, but they mey think some things are more important 
than others... 

56. Which ONE of these do you think this school believes is the most important thing for 
students? (Choose only one ) 

A. To work well vrith other people 

B. To learn the basic skills in reading, writing, arithmetic, and other subjects 

C. To become a better person 
0. To get a good job 

57. If you had to choose only the ONE most important t ling for you to learn in school, which 
would it be? (Choose only one ) 

F. To work well with other people 

G. To learn the basic skills in reading, writing, arithmetic, and other subject* 

H. To become a better person 
J. To get a good job 



In general, how do you like the following subjects? If you have never had some of these 
subjects, how much do you think you would like them? 



Like 
Very 
Much 

A. 

F. 

A. 
F. 
A. 



58 English 

59. Mathematics 

60. Social studies fh^tnrv* apoaraohv- 
government, etc.] 

61. Science 

62. Computer Education 

63. The Arts (art, crafts, music, drama, 
dance, creative writing, film- 
making, photography) F. 

64. Foreign Language A. 

65. Vocational /Career Education (shop, 
business education, home economics, 
etc.) F. 

66. Physical Education A. 

67. In general , how often do you do your homework? 

F. All of the time 

G. Most of the time 

H. Sometimes 
J. Seldom 

K. Never 



Like 
Somewhat 

B. 
G. 



B. 
G. 
B. 

G. 
B. 

G. 
B. 



Undecided 
C. 
H. 

C. 
H. 
C. 

H. 

C 

H. 
C. 



Dislike 
Somewhat 

D. 

J. 

D. 
J. 
D. 

J. 
0. 

J. 
0. 



Dislike 
Very Mich 

c 

K. 

E. 
K. 
E. 

K. 
E. 

K. 
E. 
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68. How often do your parents or other family members help you with your school work? 

A, All of the time 

B» Most of the time 

C. Only sometimes 

D. Seldom 

E. Never 



Listed below are four ways students can work in a classro om. Choose the letter on the answer 
sheet that tells how much you like or would like to work' in each w^y, even if you don't do so 
now. 

Like 





Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


69. Alone by niyself 


F. 


G. 


H. 


J. 


K. 


70. With the whole class 


A. 


B. 


C. 


D. 


E. 


71. With a small group of students, 
who know as much as I do 


F. 


G. 


H. 


J. 


K. 


72. With a small group of students, 
some who know less, some who know 
as much, and some who know more 
than I do 


A. 


B. 


C. 


D. 


E. 


Listed below are some things that might be used in a class. Choose the letter on the answer 
sheet that tells how much you like or would like to use each thing, even if you don't use it in 
a classroom. 




Like 
Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


73. Textbooks 


F. 


G. 


H. 


J. 


K. 


74. Other books 


A. 


B. 


C. 


D. 


E. 


75. Work sheets 


F. 


G. 


H. 


J. 


K. 


76. Films, filmstrips, or slides 


A. 


B. 


C. 


D. 


E. 


77. Games or simulations 


F. 


G. 


H. 


J. 


K. 


78. Newspapers or magazines 


A. 


B. 


C. 


D. 


E. 


79. Tape recordings or records 


F. 


G. 


H. 


J. 


K. 


80. Television/video 


A. 


B. 


C. 


D. 


E. 


81. Calculators 


F. 


G. 


H. 


J. 


K. 


82. Globes, maps, and charts 


A. 


B. 


C. 


D. 


E. 
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83. Animals and plants 

84. Lab equipment and materials 

85. Computers 



Like 
Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Yery Much 


F. 


G. 


H. 


J. 


K. 


A. 


B. 


C. 


D. 


E. 


F. 


G. 


H. 


J. 


K. 



Listed below are some things that you might do in a class. Choose the letter on the answer 
sheet that tells how much you like or would like to do each thing, even if you don't do it in 
cl ass. 



Like 
Very 
Much 



Like 

Somewhat 



Undecided 



Dislike 
Somewhat 



pi an 



Dislike 
Very Much 



ftfi 1 i ^fpn tn thp tparhpr 


A. 


B. 


C. 


D. 


E. 


87. Go on field trips 


F. 


G. 


H. 


J. 


K. 


88. Do research and write reports, 


A. 


B. 


C. 


D. 


E. 


89. Listen to student reports 


F. 


G. 


H. 


J. 


K. 


90. Listen to speakers vrfio come to class 


A. 


B. 


C. 


D. 


E. 


91. Have class discussions 


F. 


G. 


H. 


J. 


K. 


92. Build or draw things 


A. 


B. 


C. 


D. 


E. 


93. Do problems or write answers to 
questions 


F. 


G. 


H. 


J. 


4 

K. 


94. Take tests or quizzes 


A. 


B, 


C. 


D. 


E. 


95. Make films or recordings 


F, 


G: 


H, 


J. 


K. 


96. Act things out 


A. 


B. 


C. 


D. 


E. 


97. Read for fun or interest 


F. 


G. 


H. 


J. 


K. 


98. Read for information 


A. 


B. 


C. 


D. 


E. 


99. Interview people 


F. 


G. 


H. 


J. 


K. 


100. Do projects or experiments that 
are already planned 


A. 


B. 


C. 


D. 


E. 


101. Do projects or experiments that I 













G. 



H. 



J. 



K. 



Questions About the Learning Resource Center (LRC) 

102. Have you heard of the Learning Resource Center? 

A. yes 

B. no 
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Questions About the Career Magnet School 



Strongly Strongly 

Agree Agree Undecided Disagree Disagree 

103. I understand what the Career Magnet 

School program is trying to do. F. G. K. J. K. 

104. I would like more information about the 

Career Magnet Schools. A. B. C. D. E. 
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Attachment 3 



High School-CSEUCLA COLLABORATION 

Working Group Meeting 
February 22, 1984 

Suggested Topics for Consideration 

1. Identify the kinds of information teachers, counselors, and building 
administrators view to be useful for their owa work (student-level, 
class-level, school-level and program-level decisions). 

2, Identify what specific problems ( at any level) Royal High Staff 
would expect the information system to help them address, 

3. Ascertain level of understanding of the computerized information currently 
available to school staff and the services that can be provided, 

4, Discuss the extent to which the system meets current and anticipated needs. 
5/ Determin what additional information may be necessary to augment the system 

6. Develop methods for collecting any additional information. 

7. Identify the information that will be useful for characterizing the 
functioning and impact of the Career Magnet School (or other programs). 
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Brief Project Summary 

A Comprehensive Information System for 
Local School Improvement 

A Collaborative Research Project 

Between High School , 

Valley Unified School District, and 
UCLA, Graduate School of Education 

A lot of activity directed toward the development and use of 
information systems in school improvement efforts has been initiated 
over the past several years. Most of the developments thus far have 
occurred at the elementary school level, and most of these have been 
limited to the monitoring of test data. 

The idea that schools, both elementary and secondary, need informa- 
tion above and beyond ordinary achievement outcomes to effect decisions, 
take actions, and evaluate improvement efforts is almost self-evident. 
While test scores certainly serve diverse functions in school, they are 
merely one set of indicators of how the culture of the school functions 
and, as such, are best understood within the school context through 
knowledge of the motivations, interests, and activities of the school 
constituencies. 

Many concerns on the part o* teachers and administrators at the 
class and school levels, and staff at the district level, can be envi- 
sioned that can be significantly informed through the- judicious collection 
and analysis of pertinent information such as: attendance, student 
programs, tracking structures, student transiency, participations in 
extra curricular activities, class sizes and compositions, instructional 
activities, and the opinions and attitudes of students, teachers, 
parents and administrators. 
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We are studying these and other emerging questions by examining how 
information use becomes part of the decision-making and change processes 
within a school. Our collaborative work with High School has 

provided the perfect opportunity for reality-testing the idea since the 

Valley Unified School District has already begun to accumulate a 
diverse set of data on its students and programs using a computerized 
information system. 

The work we are currently undertaking is a project of both the 
Center for the Study of Evaluation and the Laboratory in School and 
Community Education (via THE PARTNERSHIP), units of UCLA's Graduate 
School of Education. Current and anticipated activities include (but 
are not limited) to the following: 

A. Instituting the kinds of data collection activities that may be 
necessary to establish the system's comprehensiveness (e.g., 
ongoing and pre- and post-graduate student surveys, teacher 
and parent surveys, additional school -level data, etc.). 

B. Organizing and structuring information files in a way amenable 
to the manipulations and analyses required of the data (e.g., 
student files, teacher and parent files, aggregated files at 
class and school levels, etc.). 

C. Investigating the optimal balance of hardware, software, and 
human resources required to make the system: 

1. Efficient with respect to dc.ta entry, updating, and data 
transfer between district and school. 

2. Accessible and timely to school administrators, teachers, 
and district level staff. 

D. Developing and utilizing the simplest, but most meaningful, 
methods of data manipulation and analyses. 

E. Developing and testing report formats that maximize both 
meaning and interpretation by the intended audiences. 

F. Gathering case-study material through observations and 
interviews (of and with district and school staff) regarding 
decision-making with respect to developing, implementing and 
applying information systems. 
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Brief Project Summary -3- 

It is interesting to note that this current effort at . High 
School is the first sustained (and hopefully enduring) involvement that 
has occurred out of over two years of Valley-UCLA collaborations. 
This should not be interpreted negatively. Rather, it reinforces what 
we have experienced and learned from many school improvement efforts- 
school renewal and change is not an overnight phenomenon. Certainly 
each effort has its own story to tell, but all efforts are based upon 
considerable investments of time and energy toward the mutual building 
of trust and understanding. And there is the other ingredient deserving 
special mention—people, exceptional people (teachers and district and 
building level administrators) willing to try to overcome the day-to-day 
obstacles in the circumstances of schooling that make difficult the time 
consuming activities essential to school improvement efforts. 
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Attachment 5 

Distribution of student responses to Spring Survey 
reported in terms of percentages completing the survey 

Questions About Yourself 

1. Sex: 

49 A. Male 
51 B. Female 

2. Besides English, what other languages are spoken in your home: 

77 F. None 

10 G. Spanish 

1 H. Vietnamese 

1 J. Chinese 
10 K. Other 

3. Living situation: 

78 A. With two parents (includes stepparents) 

15 B. With one parent only (mother or father only) 
3 C. Guards an(s) /foster parents 
1 D. Alone or with friends 

3 E. Other 

4. About how many hours a week do you usually spend working on a job during the school year? 

50 F. tone. I am not employed during the school year. 
Tf G. About 10 hours or less 

18 H. About 15 - 20 hours 

13 J. About 20 - 30 hours 

HE 

5. How many hours do you watch television each day? 

14 A. None 

38 B. About 1 hour 

35" C. About 2-3 hours 

Tf D. About 4-5 hours 

4 E. More than 5 hours 

Choose the ONE answer that best completes each of the following sentences. 

6. If I could do anything I want, I would like to: 

3 F. Quit school as soon as possible. 

19 G. Finish high school. 

22 H. Go to trade/technical school or junior college. 

50 J. Go to a 4-year college or university. 

"7T K. Don't know. 



7. I think ny parents would like me to : 



1 


A. 


Quit school as soon as possible. 


19 


B. 


Finish high school. 


15 


C. 


Go to trade/technical school or junior college. 


62 


D. 


Go to a 4-year college or university. 




E. 


Don't know. 
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8. Actually, I will probably : 

1 F. Quit school as soon as. possible* 
23 G. Finish high school. 

30 H. Go to trade/technical school or junior college. 
40 J. Go to a 4-year college or university. 
TT K. Don't know. 

9. How comfortable do you feel about choosing a future career goal at this point in your life? 
10 A. Very Uncomfortable 

13 B. Uncomfortable 

34 C. Neither Uncomfortable or Comfortable 

3T D. Comfortable 

TZ E. Very Comfortable 

The following sentences describe som* of the wc^ys in which people might think about themselves. 

Read each of the following sentences carefully and mark the letter on the answer sheet that 
tells how much it is like you . 

Look at the following practice sentence and mark the letter on the answer sheet that tells how 
much you agree or disagree with the sentence. 

PRACTICE Strongly Mildly Not Mildly Strongly 

Agree Agree Sure Disagree Disagree 

I am good at art * A. B. C. D. E. 

If you Choose "Strongly Agree," you're saying that you are very good at art. If you choose 
"Mildly Agree," you're saying that you are OK at art. If you choose "Mildly Disagree," you're 
saying that you are not too good at art. If you choose "Strongly Disagree," you're spying that 
you are very poor at art. 

Strongly Mildly Not Mildly Strongly 
Agree Agree Sure Disagree Disagree 



10. 


I'm popular with kids my om age. 


21 


52 


20 


5 


1 


11. 


Kids usually follow my ideas. 


12 


47 


29 


9 


3 


12. 


Most people are better liked than I am. 


6 


13 


32 


29 


19 


13. 


It is hard for me to make friends. 


4 


11 


5 


27 


52 


14. 


I have no real friends. 


3 


4 


4 


10 


79 


15. 


I'm not doing as well as I'd like to in school. 


36 


32 


5 


14 


12 


16. 


lama good reader. 


39 


37 


11 


8 


5 


17. 


I'm proud of my school work. 


16 


37 


17 


19 


11 


18. 


I'm good at math. 


22 


33 


14 


17 


14 


19. 


I'm doing the best work that I can. 


14 


28 


13 


28 


16 


20. 


I am able to do school work at least as well as 














other students. 


46 


32 


14 


6 


2 
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21. My grades are not good enough. 

22. I'm always making mistakes in my school work. 

23. I am a good writer. 



Strongly Mildly Not Mildly Strongly 



Agree 


Agree 


Sure 


Disagree 


Disagree 


27 


32 


8 


18 


15 


5 


16 


16 


40 


23 


21 


■ 38 


21 


14 


7 



Questions About Your School Life 
How much do the following words describe most of the teach er s at this school? 





Very 


Pretty 


Some- 


Only A 


Not at 




Much 


Much 


what 


Little Bit 


All 


24. Friendly 


11 


51 


27 


8 




25. Helpful 


12 


48 


28 


9 


3 


26. Have high hopes for us 


12 


28 


36 


18 


7 


27. Talk to us 


18 


39 


27 


12 




28. Let us talk to them 


17 


37 


29 


13 


A 


29. Care about us 


9 


31 


36 


16 


7 


30. Do a good job 


12 


49 


26 


8 


A 

*T 


How much do the following words describe how you feel about most of the students at this 




school? 














Very 


Pretty 


Some- 


Only A 


Not at 




Much 


Much 


what 


Little Bit 


All 


31. Friendly 


13 


51 


28 


7 


2 


32. Helpful 


7 


32 


40 


17 


4 


33. Have high hopes 


8 


28 


43 


16 


4 


34. Smart 


7 


41 


41 


9 


2 


35. Talk to each other 


48 


36 


12 


3 


1 


36. Care about each other 


17 


41 


29 


10 


3 


37. Competitive 


41 


32 


20 


5 


2 
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38. "The most popular students in this school are: (Choose only one answer) 
48 F. Athletes 
3 G. Smart students 
9 H. Members of student government 
35 J. Good-looking students 
3 K. Wealthy students 

Indicate whether or not you participate in the following activities at school. (Answer yes or 
no for each of the following). 

Yes No 



39. I participate in sports teams/drill team/flags/cheerleading. 37 60 

40. I participate in student government. 

41. I participate in music, band, drama, or other arts. 

42. I participate in honor society. 

43. I participate in school clubs/community service activities. 

Below is a list of things which may be problems at this school. Hew much do you think each is a 
problem at this school? 



8 88 

17 79 

19 77 

26 71 





Not a 


Minor 


Major 


Problem 


Probl em 


Probl ei 


44. Student misbehavior (fighting, stealing, gangs, truancy, etc.) 


17 


62 


19 


45. Poor courses or not enough different subjects offered 


40 


40 


17 


46. Prejudice/Racial conflict 


66 


26 


7 


47. Drugs 


16 


49 


34 


48. Alcohol 


18 


45 


36 


49. Poor teachers cr teaching 


33 


43 


17 


50. School too large/classes overcrowded 


59 


31 


9 


51. Teachers don't discipline students. 


57 


34 


8 


52. Poor or not enough buildings, equipment, or materials 


41 


38 


19 


53. The principal and other people in the office who run the school 


32 


34 


32 


54. Poor student attitudes (poor school spirit, don't want to learn) 


23 


51 


26 


55. Too many rules and regulations 


21 


35 


43 


56. How the school is organized (class schedules, not enough time 








for lunch, passing periods, etc.) 


12 


28 


58 
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Issues and Problems : 

Read each one of the following sentences carefully and choose the letter that tells how much you 
agree or disagree with what it says. CHOOSE ONLY ONE LETTER for each sentence. Please raise 
your hand if you have any questions. 





Strongly 
Agree 


Mildly 
Agree 


Not 
Sure 


Mildly 
Disagree 


Strongl 
uisagre 


57. What I'm learning in school is useful 

TOr rmat 1 W 1 1 1 llccU UJ KlKJW NUnt 


21 


40 


16 


16 


7 


58. What I'm learning in school will be 
useTUi Tor wnax 1 wi 1 1 neeu tu miuw 
LATER 1n life. 


34 


33 


15 


10 


8 


59. 1 think students ot aitTerent races or 
colors should go to school together. 


67 


11 


7 


3 


10 


60. Girls get a better education than boys 

at" "Hiiq vhnnl _ 


5 


6 


23 


11 


55 


61. There are places at this school where 

T rlnn 1 +■ nn hn/*aitca T'flTI a"fV*aiH n"f rvHlPP 
1 UUn u yu UcCaUbc * hi ai 1 a 1 u ui uuici 

students* 


6 


8 


6 


10 


68 


fi9 Stave net a hpttAr Pduratvion than Girls 

at this school. 


3 


3 


23 


12 


59 


63. I do not have enough time "to do my school 
work. 


15 


28 


13 


26 


18 


64. High school students should have job 
experience as part of their school 
program. 


32 


27 


23 


11 


7 


65. Many students at this school don't 
care about learning. 


22 


34 


24 


15 


4 


66. Average students don't get enough 
attention at this school. 


17 


29 


29 


17 


6 


67. Some of the things teachers want me to learn 
are just too hard. 12 




17 


?Q 


20 


68. Too many students are allowed to 
graduate from tills school without 
learning very much. 


19 


22 


23 


17 


16 


69. If I had ny choice, I would go to a 
different school. 


11 


8 


21 


18 


42 


70. There are tilings I want to learn 
about that this school doesn't 
teach. 


29 


24 


18 


15 


13 


71. It's not safe to walk to and from 
school alone. 


5 


8 


11 


15 


60 
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Strongly 
Agree 


Mildly 
Agree 


Not 
Sure 


Mildly 
Disagree 


Strong 
Disagn 


79 T ha\/A trnnhlp rpaHinn "Hip hnnk^ anH nthpr 

/£. 1 lldVw U UUUI C 1 CUU 1 llVj L41C UUUNj uIIVJ UwIICI 

materials in ny classes. 


7 


12 


8 


23 


52 


/o. me graces or inarms i get neip roe to icarn 
better* 


17 


29- 


25 


17 


11 


74* I like school. 


19. 


41 


14 


12 


12 


7^ Thp ny*aHpc n*» markc T npf" "in cIaqq havp 
ihc aucj ui iiiui n j i yet in uiqjj nave 

nothing to do with vAiat I really know. 


21 


25 


19 


21 


14 


/O. 1 nave \Jj l cam uiwiyb wimiuul Miurfuiy riiyr. 


18 


27 

(mi 


21 

Cm JL 


19 


13 


77. Parents should have a say In what is 

■(•aiinM* at* "f*h"i c c/*hnnl 


17 

JL/ 


27 

(ml 


23 


16 


16 


78. It is easy for me to get help from a 

counselor wnen planning niy buuooi proyicuii. 


Oj 


CO 


10 

JLVJ 


11 

JLJL 


9 


79. We are not given enough freedom in choosing 

nil** rlsccoc 
OUr Uldbbcbt 


27 

(mi 


21 


11 


23 


18 


80. We are not given enough freedom in 

f*hnnci nn nil** toa/'Viov^c* 
LNUUblMy UUi tcaurvib • 


49 


19 


8 


12 


10 


81. If I have a personal problem, it would be 

paw fnr nv* tn opt hpln from a counQplnr- 


19 


17 


26 


14 


23 


82. If you don't want to go to college, this 

Qphnnl Hop^n't think voii'rp VPrv "iniDort/infc* 


8 


16 


31 


21 


22 


83. Students should have a say in vtoat 1s 
taught at this school. 


37 


32 


14 


9 


6 


84. A person is foolish to keep going to 
Qrhnnl If hp/she can apt a 1oh- 

Dvllww 1 • • 1 iW J 1 IV- wUII y U w U v)"" * 


4 


4 


9 


16 


65 


85. If I need help planning for a career, it 
would be easv for me to Qet helD frcm a 
counselor. 


35 


26 


18 


11 


8 


OU. 1 1 IMS WKT YtQj 141 lb bVliUUI lUUNo. 


14 


42 


18 


16 


9 


87. It is easy to get books front the 
school library. 


40 


36 


11 


c 
0 


A 

4 


88. Things in the school library are useful 
to me. 


32 


41 


15 


7 


4 


89. Materials in the Career Guidance Center (CGC) 
are useful to me. 29 


27 


29 


8 


5 
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Questions About Teaching, Learning & Classroom Work 

All schools teach pretty much the same things, but they may think some things are 
more important than others. . . 

90. Which ONE of these does this school think is the most important thing for students? (Choose 
only o ne) 

7 F. To work well with other people , . . . . 

65 6. To learn the basic skills in reading, writing, arithmetic, and other subjects 
T3" H. To become a better person 
10 J. To get a good job 

91. If you had to choose only the ONE most Important thing for you , which would it be? (Choose 
only one) 

14 A. To work well with other people 

24 B. To learn the basic skills in reading, writing, arithmetic, and other subjects 
32 C. To become a better person 
|? D. To get a good job 

In general, how do you like the following subjects? 





Like 








Dislike 




Very 


Like 




Dislike 




Much 


Somewhat 


Undecided 


Somewhat 


Very Much 


92. English 


23 


45 


10 


14 


6 


93. Mathematics 


25 


34 


10 


14 


16 


94. Social studies (history, geography, 






13 


16 




government, etc.) 


20 


31 


16 


95. Science 


23 


30 


16 


14 


14 


96. Computer Education 


28 


27 


33 


6 


5 


97. The Arts (art, crafts, music, drama, 
dance, creative writing, film- 


















8 




making, photography) 


40 


26 


20 


5 


98. Foreign Language 


13 


26 


24 


16 


21 


99. Vocational /Career Education (shop, 












business education, home economics, 






29 


8 




etc.) 


24 


30 


5 


100. Physical Education 


43 


28 


11 


8 


8 
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101. How many hours of homework do you have each day? 
14 A None 

40 B. About 1 hour 

35" C. About 2-3 hours 
T D. About 4-5 hours 
3 E. More than 5 hours 

102. In general , how often do you do your homework? 

21 F. All of the time 

41 G. Most of the time 
2T H. Sometimes 

11 J. Seldom 
3 K. Never 

103. How soon do teachers usually return your work? 

12 A. the next day 
29 B. 2 days later 
24" C. 3 days later 
TO" D. 4 days later 

22 E. 5 days later or more 

104. When you make mistakes in your work, how often do teachers tell you how to do it correctly? 
10 F. All of the time 

35 G. Most of the time 
2"8" H. Only sometimes 
18" J. Seldom 

6 K. Never 

105. How often do your parents or other family members help you with your school work? 

7 A. All of the time 
16 B. Most of the time 
25 C. Only sometimes 
28" D. Seldom 

2T E. Never 

Listed below are four ways students can work in a classroom . Choose the letter on the answer 
sheet that tells how much you like or would like to work in each way, even if you don't do so 
now. 



106. Alone by myself 

107. With the whole class 

108. With a small group of students, 
who know as much as I do 

109. With a small group of students, 
some who know less, some who know 
as much, and some who know more 
than I do 



Like 
Very 
Much 

20 
21 

39 



31 



Like 

Somewhat 

35 
41 

35 



31 



Undecided 

11 
14 

12 



17 



Dislike 
Somewhat 

20 
15 



Dislike 
Very Much 

12 
7 



11 
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Listed below are some things that might be used in a class. Choose the letter on the answer 
sheet that tells how much you like or would like to use each thing > even if you don't use it in 
a classroom. 



Like 





Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


110. Textbooks 


11 


38 


14 


22 


12 


111. Other books 


10 


39 


26 


15 


6 


112. Work sheets 


14 


40 


15 


17 


10 


113. Films, filmstrips, or slides 


43 


35 


9 


6 


3 


114. Games or simulations 


39 


29 


16 


7 


4 


115. Newspapers or magazines 


18 


37 


23 


12 


5 


116. Tape recordings or records 


21 


28 


22 


16 


8 


117. Television/video 


54 


31 


6 


3 


1 


118. Calculators 


38 


34 


15 


5 




119. Globes, maps, and charts 


20 


34 


20 


13 


9 


120. Animals and plants 


35 


33 


17 


5 


4 


121. Lab equipment and materials 


37 


30 


14 


9 


5 


122. Computers 


48 


25 


14 


3 


5 


Listed below are some things that you might do in a class, 
sheet that tells how much you like or would like to do each 
class. 


Choose the letter on the 
thing, even if you don't 


answer 
do it in 




Like 
Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Mich 


123. Listen to the teacher 


17 


46 


13 


13 


6 


124. Go on field trips 


60 


23 


8 


3 


2 


125. Do research and write reports, 
stories, or poems 


10 


24 


13 


22 


24 


126. Listen to student reports 


10 


26 


20 


22 


17 


127. Listen to speakers who come to class 


30 


40 


11 


9 


5 


128. Have class discussions 


40 


32 


11 


7 


5 


129. Build or draw things 


29 


28 


18 


12 


8 


130. Do problems or write answers to 
questions 


11 


31 


20 


20 


13 
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Like 

Vorv 
ici jr 

Much 


Like 

Somewhat 


Undecided 


Di si ike 
Somewhat 


Dislike 
Very Much 


loii lake u=sus ur quizxcb 3 


25 




15 




27 

Cmi 


23 


1*39 Mal/Q "Pnlmc n** pppnfvHnnQ *?L\ 

10£. riaNC IIIIMb Ui icv^UiUlliyb t.*T 


23 




29 




11 


8 


133. Act things out 19 


22 


■ 


22 




16 


16 


134. Read for fun or interest 38 


31 




13 




8 


5 


135. Read for Information 17 


36 




17 




17 


8 


136. Interview people JL7 


24 




24 




17 


12 


137. Do projects or experiments that 

are already planned 20 


37 




17 




13 




138. Do projects or experiments that I 

plan 24 


30 




21 




12 


8 


Please indicate how important each of the following items was in your choice 
of classes here at High School. 




Very 
Important 


Important 


Not 
Sure 


Not 
Important 


Very 

Unimportan 


139. Taking classes from teachers I like 


58 


23 




6 


6 


2 


140. Being in the same classes as ny friends 


32 


33 




11 


15 


3 


141. Completing graduation requirements 


74 


12 




3 


3 


2 


142. Learning skills for a future job 


60 


24 




6 


2 


2 


143. Taking classes that will help me be a better 
person 


46 


31 




10 


4 


2 


144. Being challenged by taking hard subjects 


22 


33 




19 


13 


6 


145. Taking classes that will prepare me for the 
future 


55 


26 




8 


2 


2 


146. Getting a wide variety of classes 


34 


32 




17 


7 


2 


147. Preparing for college 


48 


24 




13 


5 


3 


148. Taking classes requiring little work 


10 


18 




29 




15 


149. Avoiding subjects I don't like 


22 


24 




22 


17 


7 

I 


150. Taking classes that are popular 


9 


17 




28 


27 


12 


151. Taking classes ny parent(s) consider 














important 


9 


28 




22 


21 


13 


152. Taking classes where I can get good grades 


22 


30 




18 


17 


5 
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Questions About the Learning Resourc e C enter (LRC) 



153. Have you heard of the Learning Resource Center? 
79 A* yes 

TF B. no 

154. If yes, how often have you gone with your classes to the Learning Resource Center? 
22 F. Never 

52 6. Only once or twice 

TO" H. About once or twice a month 

3 J. About once or twice a week 

2 K. Almost every day 

155. How often have you gone to the Learning Resource Center by yourself? 
50 A. Never 

25* B. Only once or twice 
8 C. About once or twice a month 

3 D. About once or twice a week 
3 E. Almost every day 

If you have ever used the Learning Resource Center, have you used any of these services? 
(Answer yes or no for each of the following). 





Yes 


" No 


156. Diagnostic testing for reading and math problems 


8 


72 


157. Entry testing for proper class placement 


8 


72 


158. Assistance with assignments from classroom teacher 


16 


64 ■ 


159. Work on tasks assigned by the Learning Resource Center 


10 


70 


160. After school seminars 


8 


71 


161. Study hall 


17 


64 


162. SAT preparation 


10 


70 


163. Proficiency test preparation 


11 


70 


164. Use the computer 


18 


62 


165. Study skills 


19 


61 


166. Language laboratory 


11 


69 


167. Assistance in researching or typing papers 


12 


68 


168. Use the typewriter 


8 


72 


169. Receive individual tutoring 


6 


73 


170. Develop library/research skills 


9 


11 


171. Develop reading skills 


9 


70 


172. Develop writing skills 


10 


69 


173. Develop math skills 


6 


73 


174. Develop listening skills 


12 


& 


175. Develop test taking skills 


14 


65 
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176. Have you received credit for Writing I through the Learning Resource Center? 
4 F. yes 

76 G. no 

177. Have you received credit for Developmental Reading through the Learning Resource Center? 
4 A. yes 

76 B. no 



Strongly • 
Agree 


Agree 


Undecided 


Disagree 


Strongly 
Disagree 


178. The Learning Resource Center is 

helping students at Royal. 16 


32 


32 


4 


3 


179. Most students know about the resources 
available In the Learning resource 
Center. 9 


23 


23 


21 


9 


18Q. I have been helped by the services of 

the Learning Resource Center. 9 


18 


20 


18 


20 


181. I am comfortable about using the services 
of the Learning Resource Center. 11 


18 


35 


11 


1 1 
11 


182. My work in the Learning Resource Center 
has helped me in ny courses. 7 


12 


32 


16 


18 


183. My work in the Learning Resource Center 
has made me feel more secure about my 
ability to do the work Assigned by ny 
teachers. 6 


11 


32 


16 


19 


Questions About the Career Magnet School 






Strongly 
Agree 


Agree 


Undecided 


Disagree 


Strongly 
Disagree 


184. I understand what the Career Magnet 

School program is trying to do. 20 


26 


18 


11 


13 


185. I would like more information about the 
Career Magnet Schools. 28 


24 


20 


6 


9 
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Attachment 6 



Distribution of student responses to Incoming Sophomore Survey 
reported in terms of percentages completing the suvey 



Questions About Yourself 

1. Sex: 

51 Male 
59 Femal e 

2. Besides English, what other languages are spoken in your home: 

82 None 

8 Spanish 

1 Vietnamese 
0 Chinese 

8 Other 

3- Living situation: 

86 With two parents (includes stepparents) 

IT With one parent only (mother or father only) 

2 Guardi an(s) /foster parents 

0 Alone or with friends 

1 Other 

4. About how many hours a week do you usually spend working on a job during the school year? 

81 Hone. I am not employed during the school year. 

TT About 10 hours or less 

4 About 15 - 20 hours 

1 About 20 - 30 hours 

1 More than 30 hours 

5. How many hours do you watch television each day? 

4 None 

23 About 1 hour 

52 About 2-3 hours 
UT About 4-5 hours 

6 tore than 5 hours 

Choose the ONE answer that best completes each of the following sentences. 

6. If I could do anything I want, I would like to: 

2 Quit school as soon as possible. 
15 Finish high school. 

13 Go to trade/technical school or junior college. 

59 Go to a 4-year college or university. 

IT Don't know. 

7. I think my parents would like me to: 

0 Quit school as soon as possible. 

17 Finish high school ♦ 

12 Go to trade/technical school or junior college, 

67 Go to a 4-year college or university. 

4 Don't know. 
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8. Actually, I will probably: 

1 Quit school as soon as possible. 

22 Finish high school. 

16 Go to trade/technical school or junior college. 

54 Go to a 4-year college or university. 

~T Don't know. 



9. How comfortable do you feel about choosing a future career goal at this point in your life? 

9 Very Uncomfortable 
16 Uncomf ortabl e 

38 Neither Uncomfortable or Comfortable 
27 Comfortable 
10 Very Comfortable 

The following sentences describe some of the ways in which people might think about themselves. 

Read each of the following sentences carefully and mark the letter on the answer sheet that 
tells how much it is like you. 

Look at the following practice sentence and mark the letter on the answer sheet that tells how 
much you agree or disagree with the sentence. 

PRACTICE 
I am good at art 

If you Choose "Strongly Agree," you.' re saying that you are very good at art. If you choose * 
"Mildly Agree," you're saying that you are OK at art. If you choose "Mildly Disagree, .you re 
saying that you are not too good at art. If you choose "Strongly Disagree," you re saying that 
you are very poor at art. 



Strongly 
Agree 


Mildly 
Agree 


Not 
Sure 


Mildly 
Disagree 


Strongly 
Disagree 


A. 


B. 


C. 


D. 


E. 





Strongly 


Mildly 


Not 


Mildly 


Strongly 




Agree 


Agree 


Sure 


Disagree 


Di sagree 


10. I'm popular with kids my own age. 


15 


59 


22 


4 


1 


11. Kids usually follow my ideas. 


9 


45 


37 


7 


2 


12. Most people are better liked than I am. 


5 


19 


34 


29 


13 


13. It is hard for ma to make friends. 


5 


11 


8 


32 


43 


14. I have no real friends. 


2 


2 


4 


9 


83 


15. I'm not doing as well as I'd like to in school 




32 


11 


16 


14 


16. I am a good reader. 


32 


39 


IS 


10 


3 


17. I'm proud of my schoolwork. 


16 


40 


20 


16 


8 


18. I'm good at math. 


25 


36 


16 


15 


9 


19. I'm doing the best work that I can. 


18 


38 


14 


22 


8 


20. I am able to do schoolvrark at least as well as 


47 


31 


17 






other students. 


4 


1 
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Strongly 
Agree 


Mildly 
Agree 


Mot 
Sure 


Mildly 
Disagree 


Strongly 
Di sagree 


21. My grades are good enough. 


20 


29 


11 


19 


21 


22. I'm always making mistakes in my school work. 


4 


18 


23 


38 


17 


23. I am a good writer. 


14 


33 


31 


14 


7 



Questions About Your School Life 

24. Who do you think the most popular students in Pjyal High School are? (Choose only one 
answer) 

40 Athletes 

1 Smart students 
14 Members of student government 
42 Good-looking students 

3 Wealthy students 

Below is a list of things which may be problems at Royal High School. How much do you think 
each is a problem at this school? 







Not a 
Problem 


Minor 
Problem 


Major 
Problem 


25. 


Student misbehavior (fighting, stealing, gangs, truancy, etc.) 


15 


49 


35 


26. 


Poor courses or not enough different subjects offered 


59 


30 


11 


27. 


Prejudice/Racial confl ict 


55 


33 


12 


28. 


Drugs 


21 


36 


42 


29. 


Alcohol 


23 


37 


32. 


30. 


Poor teachers or teaching 


48. 


34 


17 


31. 


School too large/classes-overcrowded 


54 


37 


8 


32. 


Teachers don't discipline students. 


59. 


35 


5 


33. 


Poor or not enough buildings, equipment, or materials 


55 


32 


13 


34. 


The principal and other people in the office who run the school 


11 


21 


6 


35. 


Poor student attitudes {poor school spirit, don't want to Isarn) 


27 


49 


24 


36. 


Too many rules and regulations 


40 


41 


18 


37. 


How the school is organized (class schedules, not enough time 
for lunch, passing periods, etc.) 


35 


& 


23 
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Issues and Problems: 

tead e*h one of the blowing £~^!>£^££Z2£?1£: SS5T 
agree or disagree vrith what it says. CHOOSE unly uwt lci ick 
your hand if you have any questions. 

Strongly Mildly Not Mildly Strongly 
Agree Agree Sure Disagree Disagree 
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38. What I'm learning in school is useful 
for what I will need to know NOW. 

39. What I'm learning in school will be 
useful for what I will need to know 
LATER in life. 

40. I think students of different races or 
colors should go to school together. 

41. I do not have enough time to do my school 
work. 

42. High school students should have job 
experience as part of their school 
program. 

43. Some of the things teachers want me to learn 
are just too hard*. 

44. If I had my choice, I would go to a 
different school. 

45. It's not safe to walk to and from 
school alone. 

46. I have trouble reading the books and other 
' materials in my classes. 

47. The grades or marks I get help me to learn 
better. 

48. I like school. 

49. The grades or marks I get in class have 
nothing to do with what I really know. 

50. I have to learn things without knowing why. 

51. Parents should have a say in what is 
taught at this school. 

52. We are not given enough freedom in choosing 
our classes. 

53. Students should have a say in what is 
taught at this school. 

54. A person is foolish to keep going to 
school if he/she can get a job. 

55. I like the way this school looks. 
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28 


23 
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37 


21 


5 
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12 


33 


23 


22 
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4 


7 


19 


17 


53 


6 


13 


18 


21 
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11 


14 


21 


50 


31 


27 


28 


8 


7 


20 


45 


15 


12 


8 


11 


19 


22 


24 


24 


11 


21 


31. 


17 


20 


18 


26 


30 


15 


11 


14 


22 


18 


27 


19 


24 


28 


21 


15 


12 


4 


5 


13 


19 


60 


24 


38 


28 


8 


2 



Questions About Teaching, Learning & Classroom Work 

All schools teach pretty much the same things, but they m^y think some things are more important 
than others.. ♦ 

56. Which ONE of these do you think this school believes is the most important thing for 
students? (Choose only one) 

10 To work well with other people 

55 To learn the basic skills in reading, writing, arithmetic, and other subjects 
2IT To become a better person 
15 To get a good job 

57. If you had to choose only the ONE most important thing for you to learn in school, which 
would it be? (Choose only one) 

13 To work well with other people 

38 To learn the basic skills in reading, writing, arithmetic, and other subjects 

2?" To become a better person 

25 To get a good job 



In general, how do you like the following subjects? If you have never had some of these 
subjects, how much do you think you would like them? 





Like 
Very 
Much 


Like 
Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


58 English 


16 


44 


14 


20 


6 


59. Mathematics 


26 


36 


13 


16 


8 


60. Social studies (history, geography, 
government, etc.) 


25 


33 


15 


16 


11 


61. Science 


15 


37 


20 


18 


10 


62. Computer Education 


19 


32 


36 


7 


7 


63. The Arts (art, crafts, music, drama, 
dance, creative writing, film- 
making, photography) 


43 


29 


17 


6 


5 


64. Foreign Language 


12 


31 


29 


13 


15 


65. Vocational /Career Education (shop, 
business education, home economics, 
etc.) 


17 


40 


33 


6 


5 


66. Physical Education 


34 


35 


14 


8 


9 



67. In general, how often do you do your homework? 
24 All of the time 
51 tost of the time 
T3" Sometimes 

5 Seldom 

1 Never 
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68. How often do your parents or other family members help you with your school work? 

5 All of the time 
19 Most of the time 
39 Only sometimes 
25" Seldom 

9 Never 



L i St ? d i e l°! ?r e four wa/s students can work in a classroom . Choose the letter on the answer 
sheet that tells how much you like or would like to work in each way, even if you don't do so 
now* 





Like 












Very 


Like 




Dislike 


Dislike 




Much 


Somewhat 


Undecided 


Somewhat 


Very Much 


69. Alone by myself 


16 


39 


13 


18 


14 


70. With the whole class 


20 


42 


16 


15 


7 


71. With a small group of students, 












who know as much as I do 


51 


40 


13 


5 


2 


72. With a small group of students, 












sane who know less, some who know 












as much, and some who know more 












than I do 


26 


35 


25 


10 


5 


Listed below are some things that might be used in a class. 


Choose the letter on the answer 


sneer mat tells how mucn you nke or wouia like to use each thinq. even if Y n.i Hnn't „«, •>> *„ 
a classroom. 




Like 












Very 


Like 




Dislike 


Dislike 




Much 


Somewhat 


Undecided 


Somewhat 


Very Much 


73. Textbooks 


9 


42 


24 


17 


8 


74. Other books 


10 


39 


33 


13 


5 


75. Work sheets 


16 


42 


20 


15 


7 


76. Films, filmstrips, or slides 


48 


35 


8 


6 


3 


77. Games or simulations 


35 


37 


16 


A 


4 


78. Newspapers or magazines 


19 


43 


25 


9 


4 


79. Tape recordings or records 


24 


30 


24 


15 


6 


80. Television/video 


51 


34 


10 


4 


1 


81. Calculators 


38. 


38 


15 


5 


5 


82. Globes, maps, and charts 


24 


36 


21 


11 


9 
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Like 





Vppv 

Much 


Like 

U 1 IS V 

Somewhat 


Undecided 


IT! ci life 

u i s 11 Ke 
Somewhat 


l"W ci lira 

uisiiKe 
Very Much 


83. Animals and plants 


37 


33 


20 


6 


4 


84, Lab equipment and materials 


32 


37 


18 


7 


6 


85. Computers 


38 


31 


21 


5 


5 


Listed below are some things that you might do in a class. Choose the letter on the answer 
sheet that tells how much you like or would like to do each thinq. even if vou don't do it in 
class. " 




Like 
Very 
Mucn 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


86. Listen to the teacher 


21 


49 


17 


10 


3 


87. Go on field trips 


58 


31 


7 


2 


1 


88. Do research and write reports, 
stories, or poems 


11 


25 


18 


22 


24 


89. Listen to student reports 


10 


31 


27 


19 


12 


90. Listen to speakers who come to class 


28 


42 


19 


8 


3 


91. Have class discussions 


30 


36 


17 


12 


5 


92. Build or draw things 


29 


32 


23 


10 


6 


93. Do problems or write answers to 
questions 


11 


29 


31 


19 


10 


94. Take tests or quizzes 


6 


22 


18 


31 


23 


95. flake films or recordings 


23 


28 


35 


8 


6 


96. Act things out 


18 


23 


25 


19 


16 


97. Read for fun or interest 


35 


35 


18 


7 


6 


98. Read for infomiation 


13 


39 


26 


14 


8 


99. Interview people 


15 


24 


32 


17 


11 


100. Do projects or experiments that 
are already planned 


19 


40 


22 


10 


8 


101. Do projects or experiments that I 
pi an 


22 


32 


21 


15 


10 



Questions About the Learning Resource Center (LRC) 

102. tizve you heard of the Learning Resource Center? 

31 yes 
68 no 
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MAKING SENSE OUT OF COMPREHENSIVE 
SCHOOL-BASED INFORMATION SYSTEMS 

Exploring Analyses and Reporting 
Methods for School Staff 

Kenneth A. Sirotnik and Leigh Burstein 

INTRODUCTION 

This is one of a series of reports based upon our ongoing reality 
test of the ideas embodied in our concept of systemic evaluation , the 
use of comprehensive data— data including but not limited to achievement 
outcomes— to inform school improvement efforts at all levels of the 
educational enterprise. 

Over the first two years of this project, considerable conceptual 
and developmental work was accomplished that clarified the systemic 
evaluation idea and outlined the components and contents of comprehensive 
information systems useful at the building level . (See Burstein, 1983, 
1984a-c; Sirotnik, 1984b; Sirotnik, Burstein & Thomas, 1983; and Sirotnik 
& Oakes, 1981a-b, 1982.) In a nutshell, this work included these 
contributions: 

1. Developing an epistemological framework for using the knowledge 
afforded by a comprehensive information system in school 
renewal efforts. 

2. Clarifying and integrating the basic research perspectives on 
contextual appraisal and multilevel design and analysis as 
adapted to the loral school context, 

3. Clarifying the distinctions between school and district 
perspectives on information systems and on the commonalities 
and differences in the information needs of elementary and 
secondary schools. 

4. Elaborating the functions that comprehensive information 
systems serve within and between schools. 

5. Identifying the likely contents (information domains) of a 
comprehensive information system for local school improvement, 

6. Documenting the available sources of information routinely 
collected in schools, 
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7. Reviewing the routine data collection activities of a sample of 
school districts at various stages of trying to use information 
systems, 

8. Developing an extensive array of possible survey, interview and 
observational instruments designed to collect non-achievement 
data that schools might include in their information systems. 

Currently, we are "reality-testing" these concepts by gaining 

first-hand knowledge about how to develop and sustain a school-based 

information system for ongoing school improvement efforts. Essentially, 

we studying the evolution and consequences of a comprehensive information 

system in a specific high school setting by considering, in context , 

questions such as: 

• How can such systems accommodate the diverse array of information 
needs at classroom, school and district levels? 

• How can the information data bank be organized to meet a 
variety of needs on a day-to-day basis? 

• What is the optimal balance of hardware and software at building 
levels? 

• How can the system augment and stimulate processes of curriculum 
inquiry and school renewal. 

The school selected for this reality test, is a three-year senior 
high school of approximately 2,000 students located in a two-high school 
suburban district (K-12 enrollment, approximately 20,000 students) just 
outside of the greater Los Angeles area. The selection of this site is 
responsive to both national and local concerns about secondary school 
reform and reflects a setting where a beginning computer-based information 
system already exists. The school is attempting to address pressing 
school problems such as high absenteeism and drop-out rates and the need 
for curricular reform to better accommodate student diversity and 
prepare students for post-graduation activities. There is also consider- 
able interest in exploring how the computerized information system can 
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routinely serve both improvement and day-to-day school activities. 
These conditions at the high school make it ideal for studying the 
development and use of a comprehensive information system in secondary 
school improvement efforts. 

Our basic modus operandi has been (and continues to be) to work 
hand-in-hand with a selected group of teachers, administrators, and 
counselors to develop the means whereby the district's extant 
information system can be modified and used to meet the needs at the 
school level . Towards this end, we have also been in close working 
relationship with district staff, particularly in the data processing 
division, so that any changes or additions can be easily implemented 
into existing hardware and software configurations. 

Cutting across these working relationships have been these specific 
activities: 

1. Instituting the kinds of data collection activities that may be 
necessary to establish the system's comprehensiveness (e.g., 
pre-, current, and post-graduate student surveys, teachers and 
parent surveys, additional school-level data entry, etc.). 

2. Organizing and structuring information in a way amenable to the 
manipulations and analyses required of the data (e.g., student 
files, teacher and parent files, aggregated files at class and 
school levels, etc). 

3. Investigating the optimal balance of hardware, software and 
human resources required to make the system: 

a. Efficient with respect to data entry, updating and data 
transfer between district and school. 

b. Accessible and timely to school administrators, teachers 
and district-level staff. 

4. Developing and utilizing the simplest, but most meaningful, 
methods of data manipulation and analyses. Examples of the 
kinds of possible activities are: 

a. Targeted subgroup demographic descriptions. 

b. Univariate distributions of system variables. 
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c. Bivariate and multivariate cross-tabulations, 

d. Bivariate plots. 

e. Trend analyses. 

5. Developing and testing reporting formats that maximize both 
meaning and interpretabil ity by the intended audiences. 

6. Gathering of case-study material through observations and 
interviews (of and with school and district staff) regarding 
decision-making with respect to developing, implementing and 
applying information systems. 

The focus of this report is directly on activity 5 (and, perforce, on 
activity 4 as well). This report is basically organized into two 
sections detailing both the process and product aspects of developing 
and testing report formats that teachers and administrators find interpre- 
table, meaningful and useful in their day-to-day work settings. Again, 
it must be emphasized that this is a report of work in progress ; signifi- 
cant data are yet to be obtained on many utility issues concerning the 
reporting techniques discussed below. 

PROCESSES 

As already noted, much of o r jr developmental work takes place 
collaboratively with school staff. Specifically a core group of five 
teachers (representing different subject matters), the principal and 
assistant principal, and sometimes a counselor, constitutes the primary 
vehicle for project input from the school. We will refer to these 
representatives of the school and ourselves, collectively, as the "work 
group." 

Involving as many of the other 85 staff members at the high school 
in all aspects of this project would, of course, be useful— both for 
facilitating the use of the information system as an intrinsic part of 
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the school's planning process as well as providing us with more feedback 
on the objectives of the project. However, involvement of staff at this 
level of magnitude is a costly, time-consuming, and difficult task. We 
have been fortunate, for example, to be able to gain, the release time 
necessary just to involve five teachers in this effort. Thus we have, 
in part, sacrificed exploring the consequences of a full-scale 
developmental effort with total staff involvement for the expediency and 
efficiency of a concentrated, small group work effort. Nonetheless, we 
have met with the total staff on two occasions~period-by-period meetings 
and pupil-free, minimum half day— in order to explain the purposes, 
processes and outcomes of the project, enlist their cooperation in the 
survey data collection activities, and update and share with them 
interim reports and reporting techniques. Moreover, the entire staff 
will be involved in testing and feedback regarding the several reports 
discussed in the next section. 

We have also been meeting regularly with district level staff on 
basically two accounts: (1) Updating and negotiating with the assistant 
superintendent regarding the progress of the project and the material 
and resource needs as they occur; and (2) Working closely with the 
director and senior programmer in Data Processing in order to both study 
their information system's contents and capabilities and facilitate 
their processing and use of the new information (e.g., student survey 
data) collected at the school level. 

Much of the project process that has taken place at both district 
and building levels are detailed elsewhere (see 1984 Deliverable by 
Darr-Bremme) and need not be repeated here. We will now consider the 
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more specific work-related efforts relevant to generating the analyses 
and reports based on the information system. 

Initial Activities and Student Survey 

The whole rationale for, and development and implementation of, 
additional survey data to be added to the extant information system is 
specifically addressed elsewhere (see 1984 Deliverable by Burstein). 
However, to set the context for what follows, several activities need to 
be reviewed. 

Initial meetings were held with the work group to both orient all 

participants to the general purposes and scope of the project and then 

to attend to the details of these activities: 

1« Identifying the kinds of information teachers, counselors, and 
building administrators view to be useful for their own work 
(student-level, class-level, school-level and program-level 
decisions). 

2. Identifying what specific problems (at any level) that the 
school staff would expect the information system to help them 
address. 

3. Ascertaining the level of understanding of the computerized 
information currently available to school staff and the services 
that can be provided. 

4. Reviewing the contents of the extant computerized information 
system. 

5. Discussing the extent to which the system meets current and 
anticipated needs. 

6. Determining what additional information may be necessary to 
augment the system. 
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Developing plans for collecting any additional information. 



8. Identifying the information that will be useful for characteriz- 
ing the functioning and impact of other ongoing 
programs (e.g., SIP projects). 

Among other things, it was clear from these activities that the teachers 

were aware of only some of the information and reports that were possible 
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to get from the extant system, the procedures for obtaining reports were 
slow, not always responsive, and not always flexible enough for specific 
needs, and that there were much more data of potential use that were not 
already in the information system. It was decided, therefore, that 
student, teacher, and parent surveys should be designed and field tested 
in that order of priority given available time and resources. 

The next meetings of the work group centered directly around the 
task of constructing the student survey. This work was facilitated by 
the already available compendium of potential student survey items, 
pertaining to school level and classroom level issues, developed by the 
project last year (see 1983 Deliverable, Sirotnik, Burstein, & Thomas). 
Through an interactive process of dialog, sorting, sifting, prioritizing, 
revising, subtracting irrelevant items, and adding new ones, the work 
group converged on the 185-question survey in Appendix A to this report. 
This survey was administered to students in May 1984, was computerized 
and scored by the district's Data Processing department, and was subse- 
quently analyzed at UCLA for the purposes of this project. 

Analyses and Reporting Formats 

The next msjor series of work group meetings focused on the likely 
analyses and reporting formats using student survey data and other 
system data that might capture the interests and information needs of 
school staff. 

Initially, a two-day retreat was conducted wherein the work group 
pursued an in-depth descriptive analysis of the survey results based 
upon marginal response percentages (these data are printed along side 
the items in Appendix A). In addition, a semi -structured agenda was 
followed designed to engage the group to begin considering the various 




-8- 



ways in which the analyses could be visually presented. (A copy of the 
outline followed by the work group for these discussions is included in 
Appendix B.) 

In retrospect, we may have over-structured these initial meetings 
in the sense that much of what we suggested by way of decision-making 
levels and report contents subsequently appeared in trial report forms 
discussed in the next section. Nonetheless, the perspective we brought 
to the use of information was well -substantiated in our previous concep- 
tual and practical work on this project and in work done on A Study of 
Schooling (Goodlad, 1983). Moreover, we had already become quite 
familiar with the high school through our association with it over 
several years, and we knew of the staff's and administration's concern 
with problems such as student apathy, high absenteeism, and concern over 
the appropriate foci of the school's curricular emphases * 

For whatever reasons, then, the work group resonated to this 
initial foray into the possible uses of data and the several data 
displays. Moreover, in the next couple of half -day work group meetings, 
the teachers, administrators and CSE staff worked very collaboratively on 
detailing the analyses and reporting formats most likely to be used by 
the high school staff. In the first of these meetings, in fact, one 
teacher came in armed with a list of variables that she felt were 
important pieces of information to have about students assigned to any 
particular class. 

This list eventually evolved into the first of three basic informa- 
tion report ideas generated by the work group, mocked up by CSE staff, 
and brought back to, and refined by, the work group. These reports came 
to be known as the 
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• Students-At-A-Glance 

• Class-At-A-Glance 
e School -At -A-Glance 

reports. We will discuss the details of designing and producing these 
reports in the next section. Suffice it to note here that this process 
was an interative one: ideas were generated by the work group; they 
were "brought back to the shop* and developed further by CSE staff; and 
they were presented again to the group until a working consensus v/as 
reached by all involved. 

Throughout this process, connections with district level staff were 
maintained in several ways: (1) As part of our project objectives, our 
intent was to develop reporting techniques that were compatible with 
district technological capabilities and that would, in fact, be used by 
the district to produce the several report forms noted above. Thus, 
work group efforts in terms of concepts and products were generally 
within the parameters set by district capabilities, (2) Given the 
developmental nature of the project, a certain amount of exploratory 
data analytic and graphic display work needed to be done at CSE using 
computing facilities at UCLA. Thus, we secured permission from the 
district to receive and use copies of their basic information files used 
to construct their entire information system. (The district's system 
and files are briefly described in the outline contained in Appendix C.) 
(3) Several meetings with district staff were held primarily to maintain 
good will, reaffirm our commitments to the project and the high school- 
university collaborative endeavor, keep open channels of communication, 
and update staff on the progress cf the project and the increasing 
commitments of the principal and teachers to actually using the evolving 
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reporting techniques. In one of these meetings (including CSE staff, 
the assistant superintendent, the director of data processing, and the 
principal), the district commitment was secured for the production of 
student and class reports for all teachers beginning. the second semester 
of the 1984-85 school year. 

Finally, once the trial information reports were developed and we 
were certain that the district would produce them for all teachers, a 
total staff meeting was planned and held in November 1984, The goals of 
this meeting were to reiterate the purposes and scope of the project, 
share the progress to date (including student survey results and the 
several report forms planned for dissemination), gain faculty-wide input 
to the process so far, inform staff of upcoming next-steps (e.g., 
possible teacher and parent surveys, second round of student surveying), 
and enlist more teacher volunteers to join the group. Particularly 
noteworthy, this meeting was planned and conducted mostly by the five 
teachers involved in the work group. Based upon input received from 
this meeting and the report forms generated by the work group, the 
district is currently developing the software necessary to produce the 
reports within the time frame noted above. 

This process description, although brief, should be sufficient to 
set the stage for what follows, namely, a discussion of the actual 
analyses and tabulations of data determined by the work group to be 
potentially useful at the several levels of decision-making within the 
school. We will then conclude this report with a brief discussion of 
emerging issues and an outline of the final steps with respect to this 
segment of the project. 
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PRODUCTS 

There is nothing inherent in information per se that automatically 
guarantees its usefulness. Even in the case of information generated at 
a conceptual level of need— for example, the student survey produced by 
the work group— judging the utility of data actually obtained is a whole 
new ''ballgame.' 1 What seemed obviously interesting before (in "theory") 
is not always of obvious use when the data are actually at hand. This 
was one of the primary reasons for having teachers and administrators 
face the tasks of data analyst and report designer. 

The details of how work group members attended and reacted to these 
tasks are extraordinarily intersting. We have only sketched out the 
structure of this in the process discussion above. Moreover, we will 
present mostly the outcomes of these deliberations next* adding only a 
few process descriptions as necessary. In. the subsequent section, we 
will return to these process details only to support some emerging 
issues that we think deserve special mention. Again, readers interested 
in a more in-depth, process tracing should consult the 1984 Deliverable 
by Dorr-Bremme. 

As noted previously, three types of data displays were generated by 
the work group: student-, class-, and school -at-a-glance reports. We 
will discuss each of these reports in terms of their evolution, contents 
and anticipated uses. (Information pertaining to their actual use will 
be collected in the 1985 project year.) 

Student-At-A-Gl ance 

Of the various levels at which information could be of use to 
school staff, teachers seemed to gravitate almost immediately to uses at 
the individual student level, particularly as that information might 

152 



-12- 



impact upon their classroom instruction. Early on in the project, an 
interesting "tension" emerged between teachers' predispositions to 
relate, on the one hand, to individually-focused, diagnostic data and, 
on the other hand, to group or organizationally focused data (often 
aggregates of individual data). We have labelled these two perspectives 
regarding the use of information the clinical and the social orientations, 
respectively. The clinical -social theme is an important one in informa- 
tion use and we will return to it in the next section. Suffice it to 
say here that all teachers had no problems relating to the clinical 
perspective on information use--the main issues centered around what 
data to select and how to array them in a visually satisfying manner. 

The work group already had a start on discussing these issues based 
upon a district version of a student information report (Exhibit 1) and 
also a preliminary outline submitted by one of the teachers. Interest- 
ingly, not all the teachers were even aware of the availability of the 
district report. Nevertheless, all teachers reacted somewhat negatively 
towards the report in terms of unclear variable definitions, a densely 
packed, difficult to read format, and both too much information and not 
always the most useful information was included. 

Exhibit 2 shows the final test version of the student-at-a-glance 
report, designed to provide teachers with a student-by-student roster 
for each of their classes, with the information on each student given by 
the column headings and defined by the keys at the bottom of the report. 
This report was based upon a merged file created from five separate 
district files, including the student survey data. This merged file was 
then used as input to the report generator subprogram of the SPSS system 
set up to produce this particulc- report format. The district is 
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EXHIBIT 1 
District's Student Report 



uNARF 08.21413 SCH:33 GRADE : 10 SP: EC:E2 ED: 9/12/83 DOB: 6/25/68 SEX:S TCH: 
NAME:ADAY. MADYLENE MARY PO BOX : EMRG1 : BONN IE GASS PH: 805-526-349 

ADDR: 1555- AGNEW ST # EMRG2:B0MNIE NELSON PH: 805-522-326 

CITY:SIMI VALLEY ZIP:93065 DOCTR : DRUMMOND PH : 805-526-5240 RG: / / 

PHONE : 805-527-9 1 88 BPL: CINCINNATI, 0 LSCH:WM GREEN, LAWNDALE 

FR: EUGENE 0 OC: MILLWRIGHT EMP: GENERAL MOTORS, VN* 

MR: MADYLENE OC: EMP: 

FP: 213-997-51 11 EXT: 40 MP: - - EXT: RES STATUS: BOTH PARENTS 

LOCKER: SIBLINGS: 1 PLACE: 2 ETHNICITY:4 GRID: 20 PSAT Q V 

G. P. A.: 2. 25 RANK: 355 OF 747 CTBS 'A MATH: READ: LANG: SAT Q V 
A.F.D.C. :NO HANDICAP: G.A.T.E. :NO BILING :NO SPED:NO 

ID CARD: NO SCH RULES:NO SMOKE PERM: NO AUTO PERM: NO OFF CAMP:NO YR BOOK: YE 
WORK EXP: NO FREE PER: LETTER SENT: CMS 1 

VISITS: C.G.C. : EUREKA: LIBRARY: HEALTH OFF: PRINCIPAL: PSYC: 

A. S. B. : YES ATHLETICS: NO OTHER ACTIVITY:N0 GR0UPS:59 

REFERRAL INFORMATION PAGE <01> 
REF NO PERSON MAKING REFERRAL DATE PERIOD PERSON REFERRED TO 

0001 GUTHRIE 09/14/82 05 ■ BLACKBURN 

TYPE: 19 TRUANCY TARDIES DISP: 27 PARENT CONFERNCE PHONE 

0002 HIRD 09/15/82 01 FISCHER 

TYPE: 15 SMOKING * DISP: 25 SCHOOL SUSPENSION 

/ / 

TYPE: DISP: 
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SECTION: 
TEACHER: 



EXHIBIT 2 



STUDENTS- AT- A-GL A NCE 



PREPARED ON 10 OCT 84 



STUDENT STOEENT 
KUUMEEB NAHE 



C 1^49043 
C249052 
0341950 
0449274 
0542468 
0649341 
0743686 
C849048 
0942771 
1C49050 

1143177 
1244197 
1349266 
1M92 88 
'=41828 
1649291 
1749069 
1649294 
1949295 
2C49301 

2149303 
2249306 
2349167 
2446395 
2549168 
2649309 
2749172 
2843336 
2949129 
3042520 

3142793 
3243660 
3343179 
3449323 
3549109 
3641481 
3744147 
3849262 
3949329 
4043580 



ALMS. WILLIAM B 
AEEEISON. JCHN I 
ATKCOB, DANIEL K 
EAGLE?, LOIS 0 
EAKEE, MABY M 
EOTLEE. JOYCE 
C ALE SELL, THOMAS C 
C JETER, MATHE5 
CLAEK, LAEBY P 
COCK, CHEBIL 

CCCPEB, JANE L 
CBBTIS, EEtiABE N 
EAVIS, LYNN 
EOHCAN, JOHN 
EATCH, TIMOTHY 
EKEBSCN, EOBOTHY A 
EVANS, ROBIN S 
FAFHEB, DAVIE R 
FEIDfiAN, EOBERT 
FIBK, AARCN S 

BABRIS, COR LEY 8 
HAYES, CYNTHIA 
KAPLAN, HAEOLE E 
LEKIS, ANTHONY E 
MARCUS , STANLEY 
MC AETHUR, EUGENE A 
fTT.T.FR. ELIZAEEIH 
iSCAH .1 



MLLEE 
MCCEE 
PACE 



PACE, DONALD W 
EANDLE, ANN 

FCEEETSON, SHEILA 
ECSS. BICHABB H 
SANDERS, JOAN N 
SCCTT, MABICN J 
SILVERMAN, ARTHUR 
SNYEER, EVELYN G 
SIEBN, BRUCE D 
STDABI, DCNNA 
THCRPSON. SENEY L 
BALKEB, VICKI S 



G 
B 
A 
D 
E 

12 

12 
12 
12 
12 
12 
12 
12 
11 

12 
12 
12 
12 
12 
12 
12 
12 
12 
12 

12 
12 
12 

12 
12 
12 
12 
12 
12 

12 
12 
12 
12 
12 
12 
12 
12 
12 
12 
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A 


G 




0 


0 
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A 
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T 
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S 


R 


L 


T 


S 


D 


G 


H 


A 


C 


K 



6 
3 
3 
3 
4 
6 
6 
6 

6 
6 
1 
2 
6 
7 
4 
4 
1 
1 

6 
4 
5 
3 
4 
6 
3 
2 
2 
6 

6 
5 
8 
5 
4 
5 
6 
1 
3 
4 



4Y 

HS 

HS 

4Y 

2Y 

2Y 

21 

2Y 

HS 
-> 



4Y 
4Y 
4Y 
2Y 
2Y 
2Y 
HS 
2Y 
4Y 

2Y 
2Y 
2Y 

4Y 
4Y 
2Y 
2Y 
2Y 
4Y 

4Y 
HS 

2Y 
HS 
4Y 
2Y 
2Y 
2Y 
2Y 



57 
35 
17 
18 
11 
23 
30 
13 

55 
27 
7 
8 
37 
11 
31 
35 
10 
6 

35 
17 
27 
43 
10 
35 
46 
32 
30 
47 

30 
45 
118 
27 
10 
15 
56 
31 
27 
25 



33 
52 
34 
49 
80 
68 



14 

£4 
54 
67 
88 
72 



33 86 



36 
19 
66 
67 
10 
45 
41 



26 
7 
84 
94 
12 
62 
20 



3 
83 

3 

0 
80 
68 

3 
68 

35 
7 
95 
63 
2 
40 
10 



63 64 64 
£6 96 99 



62 
1 

61 
50 
83 
80 
23 
78 
16 



83 

32 
35 
92 
60 
28 
3 

28 45 



40 
3 
8 
35 
77 
70 
30 
50 



99 74 43 



8 
47 
61 
42 
54 
92 
63 
12 



0 
49 
74 
17 
73 
52 
50 
12 



37 
31 
26 
23 
40 
95 
60 
23 



1. 1 

U9 
2,6 
1.8 
2.7 

1. 6 
2.9 
1.5 
3.0 

1.4 
2.1 
3.8 
3.6 
2.4 
2.6 
2.2 

2. 1 
1.7 
4.1 

2.4 
2.5 
2.5 

3. * 2 
2.8 
1.6 
2.9 
1.6 
2. 1 

y c 

.8 
1-1 
2.1 
1.9 
1.3 
2.9 
3. 1 
3.0 
2.2 



= MISSING 
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M 
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H 
H 
H 
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H 
H 
H 
H 
H 
H 
ii 
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H 
M 
M 

H 
M 
M 
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a 
a 

m 
a 



a 

M 
H 
H 
H 
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a 



CH SCHOOL: CAEEEB flAGNET SCHOOL. 

1=PH2SICAL SCIENCE AND TECHNOLOGY 
2=INTEENATIONAL BEIATICNS 6 POLITICAL SCIENCE 
«-M^I?? SS n Tic?^?? S I B I T «f« P l5 P 9 EflI1JG VISUAL AND FINE ARTS 
i S^n2 L ?„S HYS J CAL 5 BIOLOGICAL SCIENCES 7=IIEERAL ABTS 
8=ENTEI AND ESSENTIAIS 9=DGNT kngh 

ED EXPECT : EEUCATIONAL EXPECTATION. 

CU=QUII HIGB SCHOOL HS=FINISH HIGH SCHCOL 
2Y=GO TC TBAEE/TECBNIC SCHCCL OB JONICR COLLEGE 
4Y=GO TC 4-YEAR UNIVEBSITY ?=DONT KNOW 

DAYS AES: NUMBER OF FULL DAYS ABSENT. 

CTES TEST BESULTS ABE BEPOBTED IN PEBCENTILE BANK. 

ACAE SC: ACADEMIC SELF CONCEPT. H=HIGB H=MEEIUM L=LCH 

BCME30EK : +=ALL/MCST CF THE TIME 0=5GMETIME -=SELEOM/NEVER 

JOE: E=FULLTISE(30+) H = HALFTIME (20-30) P=EARTTIHE { 10-20) N = NCNE 

ACTIVITY: NUMBEB OF EXTEACURRICULAR ACTIVITIES (1-5) . 

ERiC SCEl II^E OF SCHCCL. +=LIKE 0=NOjE r S.DRE -=DISLIKE 



+ 
+ 

0 
+ 
+ 
0 
+ 



+ 

+ 

0 

+ 
+ 



+ 
+ 
+ 
+ 



+ 
♦ 
+ 
+ 



J 

0 
E 

B 
P 
H 
H 
N 
N 
H 
N 
N 
P 



F 
P 
N 
F 
F 
F 
N 
P 
P 



N 
H 
N 
F 
N 
P 



P 
H 
N 
P 
P 
H 
P 



A 

C 
T 
I 
V 
I 
T 
Y 

3 
0 
0 
2 
2 
0 
0 
4 
0 
1 



3 
3 
3 
0 
1 
0 
0 

1 

2 



3 
2 
0 
0 
1 
0 



0 
0 
3 
2 
0 
1 
1 
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I 
K 
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S 
C 
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+ 
+ 

+ 
+ 
+ 
0 
+ 



+ 
+ 
+ 
+ 
+ 
+ 

0 
+ 



P 2 + 
H 0 - 
H 0 + 



+ 
+ 
0 



F 0 + 
P . 0 



+ 
+ 
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currently writing its own software to essentially duplicate this report 
format given any specified class section number. 

Of the thirteen pieces of information, finally selected by the work 
group, seven come from the extant district information system and six 
are based upon responses to the student survey, (Appendix 0 contains 
annotated copies of reports to show which survey items were used,) It 
is interesting to note that one of the most influential criterion for 
the selection of information had nothing to do with content. To be 
sure, the group struggled with different opinions regarding what to 
include; for example, survey items like #19, #21, #63, and #76 (see 
Appendix A) were considered along the way but were eventually eliminated 
for lack of sufficient consensus regarding their utility. Yet, the most 
restrictive criterion for selection was this: All teachers were agreed 
that all the information for classes as large as forty students should 
fit on a single 8 1/2 x 11 page, suitable to fit in their class 
notebooks, and not overly cluttered or densely packed with data. 

At first blush, this may seem to be a rather trivial issue to get 
worked up about. Nevertheless, it was a critical one, and one that we 
will discuss further. Other issues emerging in the discussions around 
student-level data concerned the misuse and abuse of information, 
confidentiality, and creating self-fulfilling prophecies about students. 
These issues will also be discussed in the next section. 

Finally, we note some of the constructive uses anticipated by the 
teachers for the student-at-a-glance report. For example, some of the 
teachers are currently experimenting with alternative forms of grouping 
practices to handle the wide range of individual ability differences in 
their classrooms. Using cooperative learning techniques, teachers need 
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to form heterogeneous ability groups. The information in the student 
report (particularly GPA and CTBS data) provide the teachers with an 
immediate basis for trial group assignments. As another example, the 
information inherent in the homework question and the questions regarding 
after school work and extracurricular activities may provide teachers 
with some basis for dealing with students having trouble or complying 
with homework assignments. Again, we will be interviewing and sjrveying 
all teachers next year regarding what use (if any) they made of these 
data. 

Class-At-A-Glance 

Teachers generally agreed that once the student-at-a-glance report 
was available, they could get a pretty good "feeling" for their class by 
"eyeball ing" the arrays of data in each column. However, there were 
additional data from the student survey, which were not necessary to see 
student-by-student, but were still useful when aggregated at the class 
level. This information generally pertained to student perceptions of 
classroom teaching and learning and preferences for various subject 
matters and instructional practices. 

Again, unanimity among the work group members was not reached 
regarding inclusion/exclusion of data for this report; nevertheless a 
working consensus was achieved on three basic sets of items: student 
preferences for different instructional grouping configurations, student 
preferences for various kinds of instructional activities, and the 
degree to which students say they like the particular subject matter of 
the class (e.g., mathematics). 

Instead of the work group determining the report format for this 
report, the CSE staff used this opportunity to get teacher reactions to 
several different report formats ranging from straightforward tabular 

ERjC 

157 



-17- 

presentation to "state-of-the-art" graphics using the SAS statistical 
system. 

Exhibits 3-5 present the range of data displays offered to the 
work group for their consideration. Exhibit 3 shows' the class-at-a- 
glance data numerically with no graphics whatsoever. Exhibits 4a - 4b 
show the same data using the SAS graphics package and special plotter. 
Three pages were required since these graphs could not be produced in 
reduced form on a single page. Since each teacher would get different 
results for each class, it made no sense to physically cut and paste and 
reduce these figures into a single page format. (Apparently, sophisti- 
cated graphics capabilities on a main frame computer are much less 
tractable than those on a microcomputer, e.g., Macintosh. See 1984 
Deliverable by Ender.) Finally, Exhibit 5 presents a compromise 
solution using cruder graphics (ordinarily available print characters), 
but in a compact, easy to read format. 

Although they had no trouble interpreting the results, teachers 
immediately rejected the format in Exhibit 3. And, although they were 
moderately impressed with the pretty graphics in Exhibits 4a-4c, they 
were once again adamantly against receiving three sheets of paper 
containing information that could fit on one page. Moreover, they 
actually liked the simplicity of the cruder graphics in Exhibit 5~thus 
their decision was quick and easy to make. 

During these discussions, issues again emerged such as the clinical- 
social orientations regarding the importance and use of information. 
Aggregating data at the class level also seemed to provoke another 
issue, namely, the reaction to data as if they were inherently directive. 
For example, does the information in the sample class report "tell" the 
teacher not to assign research projects to his/her class because it is 
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EXHIBIT 3 



Section No: 



CLASS AT A GLANCE 
Date 

No. Enrolled Sutdents: 
No. Students taking Survey: 



Like 
Very 
Much 



Like 
Somewhat 



Undecided 



Dislike 
Somewhat 



Dislike 
Very Mich 



93. Liking of mathmatics 



50 



46 



INSTRUCTIONAL GROUPING REFERENCES 

Listed below are four ways students can work in a classroom. Choose the letter on the answer sheet 
that tells how much you like or would "like to work in each way, even if you don't do so now. 



Like 



Undecided Dislike 



106. Alone by myself 

107. With the whole class 

108. With a small group of students, 
who know as much as I do 

109. With a small group of students, 
some who know less, some who know 
as much, and some who know more 
than I do 



57 
49 

69 



63 



6 
17 

11 



33 
34 

34 



29 



STUDENT ACTIVITY PREFERENCES 

Listed below are some things that you might do in a class . Choose the letter on the answer sheet that 
tells how much you like or would like to do each thing, even if you don't do it in class. 







Like 


Undecided 


Dislike 


123. 


Listen to the teacher 


57 


20 


23 


124. 


Go on field trips 


77 


9 


14 


125. 


Do research and write reports, 










stories, or poems 


26 


9 


66 


126. 


Listen to student reports 


33 


20 


43 


127. 


Listen to speakers who come to class 


74 


6 


20 


128. 


Have class discussions 


74 


11 


14 


129. 


Build or draw things 


46 


29 


26 


130. 


Do problems or write answers to questions 


40 


26 


34 


131. 


Take tests or quizzes 


31 


17 


51 


132. 


Make films or recordings 


40 


23 


33 


133. 


Act things out 


31 


23 


46 


134. 


Read for fun or interest 


54 


26 


20 


135. 


Read for information 


51 


17 


31 


136. 


Interview people 


33 


29 


34 


137. 


Do projects or experiments that 










are already planned 


51 


20 


29 


138. 


Do projects or experiments that I plan 


43 


29 


29 
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LIKING OF MATHEMATICS 



PERCENTAGE 
60 - 

50 i 

AO H 
30 H 

Oft 



10 ■! 



mm 
mm 




iOOOC<XXX?< 





LIKEMUCH LIKESOME UNDECIDE DSLKSOME 

LIKE MIDPOINT 



DSLKMUCH 



EXHIBIT 4a 
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INSTRUCTIONAL GROUPING PREFERENCES 

ALONE 



WHOLE CLASS 



HOM SMALL CLASS 



HET SMALL CLASS 



LEGEND : PREF 




10 20 30 
FREQUENCY 



LIKE 
DISLIK 



UNDECD 



EXHIBIT 4b 
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STUDENT ACTIVITY PREFERENCES 



MIDPOINT 

LISTEN TEACHER 
GO FIELD TRIPS 
DO RESEARCH ETC 
LISTEN STUDENT 
LISTEN SPEAKER 
CLASS DISCUSSION 
BUILD/DRAW THING 
DO PROBLM/ANSWER 
TAKE TEST/QUIZ 
HAKE FILM/RECORD 
ACT THINGS OUT 
READ FOR FUN 
READ FOR INFO 
INTERVIEW PEOPLE 
DO PROJECT PLNED 
DO PROJECT I PLN 

LEGEND: PREF 




10 20 30 
FREQUENCY 



LIKE 
DISLIK 



40 



UNDECD 



EXHIBIT 4c 
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EXHIBIT 5 



CLASS AT A GLANCE 
FALL 84 

SECTION NO: XXXX 

MO, ENROLLED STUDENTS: 35 

NO. STUDENTS TAKING SURVEY* 35 



INSTRUCTIONAL GROUPING PREFERENCES 

* 

ALONE XXXXXXXXXXXXXXXXXXXX — ************* 

WHOLE CLASS XXXXXXXXXXXXXXXXX ************ 

HOM SMALL CLASS XXXXXXXXXXXXXXXXXXXXXXXX ******* 

HET SMALL CLASS XXXXXXXXXXXXXXXXXXXXXX ********** 

XXX LIKE UNDECIDED *** DISLIKE 



LIKE VERY MUCH 
LIKE SOME 
UNDECIDED 
DISLIKE SOME 
DISLIKE VERY MUCH 



LIKING OF MATHEMATICS 



************************* 

*********************** 

* 

* 



STUDENT ACTIVITY PREFERENCE 



LISTEN TEACHER 
GO FIELD TRIPS 
DO RESEARCH ETC 
LISTEN STUDENT 
LISTEN SPEAKER 
CLASS DISCUSSION 
BUILD/DRAW THING 
DO PROBLEM/ ANSWER 
TAKE TEST/QUIZ 
MAKE FILM/RECORD 
hoi i niiNoo Qui 
READ FOR FUN 
READ FOR INFO 
INTERVIEW PEOPLE 
DO PROJECT PLNED 
DO PROJECT I PLN 



XXXXXXXXXXXXXXXXXXXX ******** 

XXXXXXXXXXXXXXXXXXXXXXXXXXX ***** 

xxxxxxxxx *********************** 

xxxxxxxxxxxxx *************** 

xxxxxxxxxxxxxxxxxxxxxxxxxx — ******* 

xxxxxxxxxxxxxxxxxxxxxxxxxx ***** 

xxxxxxxxxxxxxxxx ********* 

xxxxxxxxxxxxxx ********< **** 

xxxxxxxxxxx ****************** 

xxxxxxxxxxxxxx ************* 

XXXXXXXXXXX- — -if *************** 

xxxxxxxxxxxxxxxxxxx ******* 

xxxxxxxxxxxxxxxxxx *********** 

xxxxxxxxxxxxx ************ 

xxxxxxxxxxxxxxxxxx ********** 

xxxxxxxxxxxxxxx ********** 
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the least liked class activity 1 , or does the information provide a 
context for teachers in dealing with student affect, dispositions, etc*, 
when, in fact, research projects are assigned? We will return to this 
and othrr issues shortly. 

School -At-A»Glance 

Up to this point, information was treated in the context of specific 
students in specific classes, either as individual data points or as 
aggregated at the class level. In moving to the school level, a 
significant shift in orientation along the clinical to social data use 
continuum is required; all teachers must now become involved in the same 
set of data from the perspective of organizational needs, 
decision-making, planning, evaluation, and so forth. 

Interestingly, it was not until this level of information use was 
considered that administrators became noticeably more involved and 
teachers become more passive in terms of inclusion-exclusion decisions. 
Perhaps this phenomenon was due, in part, to other reasons {e.g, y the 
way CSE staff structured work group meetings) besides the obvious ones 
of traditional role and organizational structures, at least the ones 
that usually get played out in practice— teachers responsible for what 
goes on behind the classroom door; principals responsible for schooling 
issues that have visible impact at the building level. 

In any case, the school-at-a-glance report that eventually emerged 

(see Exhibits 6a - d) was largely influenced by the principal's interest 

in what he saw as the several main issues, addressed by data in the 

student survey, that were of immediate concern to the high school. 

These issues were: 

1. What are the curricular goal emphases at the high school; what 
should they be? 
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Student Survey Results 
May, 1984 



EXHIBIT 6a 
School -At-A-Gl ance 



FUNCTIONS OF SCHOOLING 



Social Development 

Instruction that helps students learn to get along with others, prepares 
students for social and civic responsibility, develops students 1 awareness 
and appreciation of our own and other cultures* 

Intellectual (Academic) Development 

Instruction 1n basic skills 1n mathematics, reading, and written and verbal 
communication and 1n critical thinking and problem solving abilities. 

Personal Development 

Instruction tnat builds self-confidence, self-discipline, creativity, and 
the ability to think Independently. 

Vocational Development 

Instruction that prepares students for employment, developing the skills 
necessary for getting a job, developing an awareness about career choices 
and alternatives. 



Sowe Student Perceptions : 
(see survey questions 90 & 91; note wording 
students could only choose one) 



SCHOOL V* 
ME 




mrq-!TTTjn rr^ mryTTlTp 

0 20 40 60 80 100 
PERCENT 



STiCTAL 
PERSONAL 



ACADEMIC 
VOCATION 




Students perceive the school as emphasizing mainly the academic function; from 
the students 1 point of view, however, they tend to spread the emphasis around to 
the other goal areas, particularly the personal and vocational functions. 



Congruency : 

35% of the students place the most Importance on the same goal 
area they see the school as emphaasizing. To put it the 
opposite way, nearly 2/3 of the students would prefer a dif- 
ferent goal emphasis than the one they perceive. 
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EXHIBIT 6b 



Page 2 



DO THESE PERCEPTIONS CHAM6E DEPENDING UPON GRADE LEVEL? 



SCHOOL 
ME 



lOtb GRADE 







■ 


1 


0 


* 







0 20 40 60 80 100 



Uth GRADE 



\ 


■ 


1 


'A 


inn: 


• Ml 



0 20 40 60 80 100 



12th GRADE 







1 


1 









20 40 60 80 100 



PERCENTAGE 



mm 



AL 



mm 



The trends, 1f any, are slight. Emphasis on Personal Development increases 
across grades (29% of 10th graders, 33% of 11th Graders and 38% of 12th graders) 
while emphasis on Social Development (16% in 10th grade, 17% in 11th grade, 11% 
in 12th grade) and Vocational Development (31% in 10th grade, 26% in 11th 
g.'ade, 25% in 12th grade) decreases. 



DO THESE PERCEPTION CHANGES DEPEND UPON SEX? 



SCHOOL 



ME 



MALE 



V. 




I 



2 





1 1 ' 1 I ■ ' ■ ■ | ' I ' I | I I I I | I I I I [ T 

0 20 40 60 80 100 



PERCENTAGE 



FEMALE 



A 



7k 




0 20 40 60 80 100 



SOCIAL 
PERSONAL 



ACADEMIC 
VOCATION 



Boys place greater emphasis on vocational development than girls (33% of boys 
versus 22% of the girls) while girls place greater emphasis on Personal 
Development than boys (37% of girls versus 29% of boys). 
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EXHIBIT 6c 



Page 3 



STUDENT ASPIRATIONS AMD EXPECTATIONS 

(Survey questions 6, /, and 8) 




PERCENTAGE 

QUIT mm HIGHSCHOOL 

2-YR. COLLEGE csrssa 4-yR, COLLEGE 
DON'T KNOW 



MAIN TREND: Half of the students would like to go to a 4 year college or 
university in contrast with only 22% aspiring to attend a 2-year college. Their 
expectations, however, drop by about ten percent; 40% expect to go to university 
and 30% expect to go to vocational school /junior college. Students perceive 
their parents' attitudes to be more 1n line with students' aspirations than with 
student's expectations. 



10th GRADE 




i i i I i i i « I i i i i I i 

60 3G 1GG 



11th GRADE 



f 
1 











T 

2Q 



40 



-i-i-P 

60 



8G 



' ' i ' 

1GG 



12th GRADE 




Eg™ 




2G 4G 60 8G 1 GG 



PERCENTAGE 

177777773 QUIT 

2-YR. COLLEGE 
DON'T KNOW 



HIGHSCHOOL 
580 4-YR. COLLEGE 



The general trend in aspirations is toward more education (both 2 year and 4 
year college) across grades while the trend 1n expectations is toward less 
four-year college and more two-year college. While the percentage of students 
aspiring to attend a four-year college increases slightly across grades (from 
48% at 10th grade to 53% at 12th grade), the percentage of students that expect 
to attend a four-year college decrease slightly (44% at 10th grade to 38% at 
12th grade). The percentage of students expecting to attend a trade school or 
junior college increases substantially across grades (22% 1n 10th grade, 30% in 
Uth grade, and 39% in 12th grade). 
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NOTE : According to district records, only 
5-7% of all graduating seniors go on to a 
4-year college. 
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EXHIBIT 6d 



Page 4 



GPA: Averages for Males and Females at Each Grade Level 
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Two slight tendencies are apparent: (1) Boys show lower GPA averages than 
girls, and (2) GPA goes down in the 11th grade. 



DAYS ABSENT: Averages for Hales and Feaales at Each Grade Level 
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Several trends are noteworthy: (1) Boys are generally absent more days than 

girls; (2) Absences increase almost linearly from the 10th through the 12th 

grades (roughly 3 to 4 more days absent in each grade level); (3) The increase 

in days absent over grade levels is more exaggerated for girls than boys (in 

fact, girls slightly surpass boys in the 12th grade). gjrg-j QQPY AVAILABLE 
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2. What do students and parents want; and how do these perceptions 
and expectations jibe with what students actually do upon 
graduation? 

3. What are the implications of these issues for the Career Magnet 
Program and student comfort in selecting a career path in this 
program? 

Basically, the data reported in Exhibit 6 begins to explore the 
information relevant to the first two issues by bringing to bear relevant 
student survey results along with several other variables from the 
district's information system. It should be emphasized that although it 
was certainly the intent of our project to capture data relevant to the 
school in these reports, we were also concerned with analysis and 
reporting issues such as: 

• the optimal balance of descriptive text and graphics. 

# the relative appeal of one graphical mode over another. 

* easy to interpret graphical techniques for representing the 
relationships between two or more variables. 

• the amount of information to be contained in any one report. 

To be sure, many graphical techniques are available and none that 
we have used thus far are particularly novel. Nonetheless, knowing 
about things like bar charts, histograms, pie charts, frequency polygons, 
and so forth is one thing, using them in certain contexts for certain 
purposes to be understood and used in those contexts is quite another 
thing. It is quite clear in the literature how well known graphical 
techniques can be totally misused, misinterpreted, and/or irrelevant to 
the purposes intended (see, for example, discussions in Horwitz & 
Ferleger, 1980; Huff, 1954; and Tufte, 1983). 

In a particularly enlightening and creative book on graphical 
methods, Tufte (1983, p. 51} outlines what he considers to be generic 
principles underlying quality visual presentation of quantitative data. 
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"Grapical excellence' 

... is the well -designed presentation of interesting data— a 
matter of substance, of statistics, and of design. 

. . . consists of complex ideas communicated with clarity, 
precision, and efficiency, 

... is that which gives to the viewer the greatest number of 
ideas in the shortest time with the least ink in the smallest 
space. 

... is nearly always multivariate. 

. . . requires telling the truth about the data. 11 

In order to achieve these principals in practice, "graphical displays 
should 



t show the data 

• induce the viewer to think about the substance rather than about 
methodology, graphic design, the technology of graphic 
production, or something else 

t avoid distorting what the data have to say 

t present many numbers in a small space 

t make large data sets coherent 

t encourage the eye to compare different pieces of data 

• reveal the data at several levels of detail, from a broad 
overview to the fine structure 

t serve a reasonably clear purpose: description, exploration, 
tabulation, or decoration 

t be closely intergrated with the statistical and verbal 
descriptions of a data set," (p. 15) 



Although we have not been particularly creative with daring new 
visual displays, we have tried to incorporate many of these principles 
while at the same time being sensitive to the needs and concerns expressed 
by administrators and teachers in the work groups. Thus, the reports 
displayed in Exhibit 6a - d reflect deliberate attempts to: 
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1. Include just enough narrative to explain the major trends 
embedded in the graph (s) and include onlv the most relevant 
numerical results upon which the graph(s) was(were) based. 

(All teachers were given the raw data for reference purposes in 
the form of the student survey in Appendix A.) 

2. Experiment with different graphical techniques that may represent 
the same data but highlight different emphases. The two graphs 
in Exhibit 6a, for example, are based on the same survey 
questions but call attention to different comparisons. The 
first graph highlights relative emphases on the schooling 
functions (social, intellectual, personal, vocational), while 

the second highlights the difference between perspectives 
(schools 1 emphasis vs. student's preference). 

3. Organize visual displays thematically, with one theme per page, 
each successive page building upon previous ones, and all pages 
adding up to a reasonable (not overly data-laden) foray into 
the issues of concern to the group. 

4. Go beyond a simple univariate treatment of information but not 
overly complicate the analytical and graphical treatments of 
data. The comparisons by grade level and sex (Exhibits 6b and 
c) and bivariate relationships in Exhibit 6d are illustrative. 

5. Bring to bear a variety of information from a variety of 
sources (e.g., student survey, extant information system, and 
district records). 

We must emphasize, agiin, that these reports are designed primarily 

as an "experiment" to test the feasibility of various data displays; as 

such, they only scratch the surface of what can be done analytically 

with the data in a comprehensive information system. In the coming 

year, we plan to more fully explore the analytical capabilities of the 

system, especially as the system will be augmented with another round 

of student survey data (for longitudinal comparisons) and possibly 

teacher and parent survey data as well. 



EMERGING ISSUES 



We have already alluded to (and in some cases, briefly described) 
some issues that seem to continually appear as teachers, administrators, 
and outside collaborators attempt to work together on designing, develop- 
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ing and using an information system responsive to needs at the building 
level. We will now briefly review and elaborate these issues. 

Clinical Versus Social Uses of Information 

Often in the deliberations over which piece of student survey data 
might be useful, particularly for class and school level reports, 
considerable differences of opinion seemed to occur between members of 
the work group. Often, the generic form of the debate seemed to take 
the forms of "I don't see how I could use this piece of information in 
teaching a student" versus "I think these data could help us (me) make 
planning decisions about the school (my class)." In effect, the disagree- 
ment was more a matter of differing orientations than it was of infor- 
mational content. 

CSE staff intervened a number of times in these discussions 
attempting to clarify the individual, diagnostic, clinical orientation, 
on the one hand, and the organizational, planning, social perspective, 
on the other. These interventions seemed to help clarify and facilitate 
the discussion and also permitted the observation that some teachers 
simply placed less value on the social use of information. Nevertheless, 
teachers could more easily sort out the substantive aspects of inclusion- 
exclusion decisions; they could, for example, agree that an item like 
"How much do you like mathematics?" had less diagnostic use at the 
individual level, yet could be aggregated at the classroom level to 
provide information helping the teacher deal with climate and learning 
environment issues. 

The clinical -social distinction is not a new one in work relating 
to assessing organizations, but its manifestation in the school setting 
as people attempt to design and make sense out of comprehensive 
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information systems deserves further study* Clearly, the issue has a 
direct bearing upon statistical and psychometric concerns arising out of 
multilevel analysis. Also clear, are the interacting effects of the 
socio-cultural context and circumstances of schooling and teaching that 
may predispose teachers to "think clinically"" we are referring here to 
features such as the hierarchical organization of schooling, traditional 
roles of administrators vis a vis teachers, and the ways teachers have 
for developing and organizing their "working knowledge," 

The clinical-social issue both cuts across the issues that follow 
and discussing it in more depth is beyond the scope of this report. As 
more data are accumulated in the 1985 project year, we expect more 
insights regarding this phenomenon. It will thus serve as the major 
focus for a 1985 project deliverable. 

Teachers As Researchers/Data Analysts 

Over the course of our meetings with the work group, teachers and 
administrators behaved more and more like trained researchers, asking 
more questions of the data and requiring more sophisticated treatments 
of the data (e.g., bivariate and multivariate analyses). Certainly part 
of this may be due to our presence and our deliberate suggestions 
regarding the ways in which data can be explored (see again Appendix B). 
However, these enlightened approaches to data exploration were clearly 
evident in the work group from the beginning and were evidenced by 
several teachers never even involved in the work group. 

Our point here is to cast some doubt on the often heard lament that 
teachers don't really care about having more and better information and, 
even if they did, wouldn't know what to do with it. The fact of the 
matter may be less an issue of caring and more one of professional 
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opportunity. The current circumstances of teaching and administrating 
simply do not permit the kind of time necessary for informed dialog, 
decision-making, action-taking, and evaluation that characterizes a 
dynamic and renewing organization (Goodlad, 1975). ' 

As the age of information explodes upon us, along with the techno- 
logy to handle it, the pressures for organizational change become 
impossible to ignore. Many organizations in the private sector have 
been both receptive and responsive to, for example, the increased role 
of workers as informed decision-makers (Peters and Waterman, 1982). Our 
belief is that schools (and their districts) will need to change in 
similar and perhaps even more profound ways to allow administrators and 
teachers to participate more fully as professionals and engage in 
inquiry processes that can be significantly advanced by the kind informa- 
tion systems we have been discussing. (See 1982 Deliverable by Sirotnik 
and Oakes for a more in-depth discussion of the inquiry and school 
renewal model being suggested here.) 

The Power of Numbers 

When teachers are ordinarily presented with quantitative data, it 
is usually of the prescriptive variety and often for the purpose of 
accountability—standardized test scores are the prime example. Even in 
ordinary, everyday life we are bombarded with numbers that seem to 
suggest responding rather than reflecting--Dow Jones averages, inflation 
rates, weather reports, etc. It is not surprising to see school staff, 
therefore, reacting to survey data as if they contain the prescription 
for educational change instead of providing just one more heuristic for 
helping to understand the possible directions for change. 
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The typical way in which we observed this phenomenon ii illustrated 
in the following exchange (paraphrased here) by members of our work 
group: 

Person A : If we allow these data to make decisions for us, then we 
must be concerned with the validity of the student responses. 

Person B : I give tests— I have a vested interest, as a teacher, in 
student assessment. Would I reconsider this method of evaluation 
just because kids say they don't like tests? Maybe so. 

Person A : I think learning to read is more important than any 
subject matter per se. So I assign reading both for content and 
skill development. If the survey indicated students don't like 
textbooks, should I not bother to teach them to read? 

Person C : It seems that the dilemma here is more a question of 
perceptions regarding what the data mean. 

Person C, of course, hit the nail on the head. We added our own "two 
cents" to this discussion by noting that data do not make decisions- 
people do. Thus, information is best used not as a blueprint for action 
but as a catalyst for, and adjunct to, staff discussion and decision- 
making. 

These kinds of discussions occurred a number of times throughout 
the course of the project, and it seemed to be of some considerable 
relief to the work group to know that it's OK to be proactive rather 
than reactive in regard to information and the use of information. 

The Quest for Simplicity in Complexity 

Although issues of content were always of importance to the work 
group as they deliberated over what and what not to include in the 
several reports, of equal (or perhaps even more) importance was the 
group's wish for simple, short, uncluttered, non-numeric displays of 
data. The prime example was the bottom line for student- and class-at- 
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a-glance reports— whatever they contained, they needed to fit on single 
8 1/2 x 11 pages. 

Not withstanding the validity of the group's concerns, schooling 
and the assessment of it is extraordinarily complex; there are multiple 
data sources, multiple domains of potential data, multiple levels at 
which information is used, multiple methods for obtaining data, and 
multiple analytic and reporting techniques. Yet given this complexity, 
we still seek simple representations of it. 

We do not mean to suggest that this is an unworthy goal or that a 

complex problem necessitates a complex solution. In fact, as a society, 

we will probably face a growing need to provide a more "human edge 11 to 

the products of an increasingly technological world. (See Naisbitt's, 

1982, analysis of the "high tech/high touch 11 megatrend as we move from 

an industrial to an information society.) Our point here is simply to 

« 

note the tension between the legitimate requirement for simplicity in 
representing the information pertaining to a complex process (schooling) 
and setting (schools). 

The Misuse of Information 

The potential for misusing information—violating confidentiality, 
creating self-fulfilling prophecies regarding individuals (e.g., students, 
based on CTBS "ability" scores), misinterpretations, overinterpretations, 
and inappropriate applications of data, and so forth— has always been a 
feature of districts and schools. The presence of a comprehensive and 
accessible information system merely exacerbates the problem. 

Nonetheless, it is still a serious problem and we have been sensitive 

to it in the general context of developing and using computerized 

i 

information systems (Sirotnik, 1984). The teachers in the work group 
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tas well as several others in the faculty at large) have also beer 
worried about the abuse of people through the misuse of data, and they 
have voiced their concern several times over the course of this project. 
Interestingly, this concern is provoked by the presence of information 
like self-concept, homework compliance, and educational expectations. 
Yet data such as the ever-present standardized test scores have always 
had as much (or more) potential (and actual) misuse—for example, the 
stigmatizing and disenfranchising from academic excellence of many 
students in low tracked classes (Oakes, in press). 

But we find no comfort in the old notion that it is people, not 
the information per se, that carry the potential for abuse. It may well 
be that the costs due to the misuse of information may well outweigh the 
advantages of individually based reports like students-at-a-glance. But 
then need all such reports be automatically generated for all teachers? 
Perhaps only those teachers requiring certain kinds of data for reasonable 
purposes could request and receive immediately reports tailored for 
their needs. We plan to study the feasibility of this alternative 
(including technological implications) in the coming year. 

Needless to say, this whole issue is bound to become messier before 
it becomes clearer. We can only remain sensitive to the misuse of data 
within the context of the constructive use of information systems. 

The Educative Function in Collaboration 

We have, we believe, been appropriately self-conscious regarding our 
role in this project. Our presence in the work group has not been 
exactly unobtrusive, nor has it been unduly interventionist. We have 
tried to walk that fine line sepdrating the role of observer-researcher 
from active participant-director. 
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Throughout the discussion in this report, we have tried to make 
clear how we may have shaped the course of events in what we hope was an 
educative, more than a directive, role. This observation is not meant 
in any way to be a condescending statement by university-based educators 
bringing their words of wisdom down to the less informed levels of 
school practitioners. The fact of the matter is that the educative 
function in collaborative research is quite reciprocal— we have been 
educated often during this project by teachers and administrators concern- 
ing the realities of schooling and the meaning and use of information in 
the context of practice. 

What all this suggests to us is the need for someone or group to 
serve in an educative and collaborative role regarding issues (like 
those emerging here) pertaining to the development and use of information 
systems. Given the trends toward increased use of technology, moves 
toward decentralization, and needs for reconfiguration of resources, it 
is not hard to imagine an FTE position at the building level explicitly 
for R&D-type activities based upon a comprehensive information system. 

FUTURE DIRECTIONS 

As stated in the proposal for FY 1985, this final project year will 
be devoted to achieving closure on the basic problems raised and studied 
in this project. The deliverables this year represent, essentially, 
beginning versions of the final reports. All these problems—organi- 
zational and process issues in developing and using information systems, 
expanding the concept of information to include more than just achievement 
test data, developing a feasible hardware-software interface that meets 
the information needs at the building level, and analyzing and displaying 
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information in clear and useful ways—are interconnected and will be 
enlightened by the activities planned for FY 1985 (see proposal). 

Of particular relevance to this report will be the data gained by 
studying how administrators and staff respond to the several reports that 
will be produced and distributed by the district. Other data collection, 
analysis, and reporting activities are also being planned for follow-up 
student surveying and teacher and parent surveys. As these activities 
proceed, not only will data be obtained regarding the actual use, 
misuse, and/or nonuse of information systems by school staff, but also, 
additional insight into the clinical-so;ial distinction raised above is 
anticipated. This issue, which we believe is embedded in many of the 
other emerging issues noted, will form the basis of a final report for 
FY 1SG5 as wel 1 . 
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APPENDIX A 



Student Survey and 
Results of Survey Conducted 
in May 1984 (N = 1461) 



High School Student Survey 



The survey you are about to complete will ask you questions about 
yourself and about your school. This is not a test. There are no right or 
wrong answers. The survey will give you an opportunity to express how you 
feel about what happens in your classes and around school. That is why it 
is important to answer the questions as truthfully and as carefully as 
possible. 



DO NOT WRITE ON THESE PAGES 

MARK YOUR ANSWERS ON THE ANSWER SHEET PROVIDED. You will notice that 
answers go from A to E or from F to K. This does not matter. Simply 
choose the one answer that best fits your opinion for each question. MARK 
ONLY ONE LETTER ON THE ANSWER SHEET FOR EACH QUESTION. For example, if you 
chose answer B for question number 5, you would mark the answer sheet like 
this: 

ABODE 

SOGOOO 

Or, if you chose answer J for question nunber 6, you would mark the answer 
sheet like this: 

FGHJK 

sooo$o 

Remember, mark only one letter on the answer sheet for each question. If 
there are any words or questions you don't understand, please raise your 
hand and ask for help. 



DO NOT BEGIN UNTIL YOU RECEIYE MORE INSTRUCTIONS 

This question will be answered differently than the othersn You will 
use the blue box at the top of the answer sheet. Read the list of Career 
Magnet Schools below. 

1. Physical Science and Technology 

2. International Relations S Political Science 

3. Business 

4. Industry 

5. Performing, Visual and Fine Arts 

6. Mental, Physical & Biological Sciences 

7. Liberal Arts 

8. Entry and Essentials 

9. Don't Know 

Now, using the last column of the blue box (to the far right), mark the 
number on the answer sheet that matches your career magnet school . 

Starting with number 1 on the survey, the rest of the questions will 
be answered in the white area of the answer sheet. Remember, do not mark 
on the survey sheets themselves. Mark one answer for each question on the 
answer sheet. 
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High Student Survey Results 
May 1984 

Questions About Yourself 

1. Sex: 

49 A. Male 
51 B. Female 

2. Besides English, what other languages are spoken in your home: 

77 F. None 

10 G. Spanish 

1 H. Vietnamese 

1 J. Chinese 
10 K. Other 

3. Living situation: 

78 A. With two parents (includes stepparents) 

15 B. With one parent only (mother or father only) 
3 C. Guardian(s) /foster parents 
1 D. Alone or with friends 

3 E. Other 

4. About how many hours a week do you usually spend working on a job during the school year? 

50 F. None. I am not employed during the school year. 
T? G. About 10 hours or less 

18 H. About 15 - 20 hours 

13 J. About 20 - 30 hours 
T 

5. How many hours do you watch television each day? 

14 A. None 

38 B. About 1 hour 

35" C. About 2-3 hours 

TT D. About 4-5 hours 

4 E. More than 5 hours 

Choose the ONE answer that best completes each of the following sentei.Jes. 

6. If I could do anything I want, I would like to: 

3 F. Quit school as soon as possible. 

19 G. Finish high school. 

22 H. Go to trade/technical school or junior college. 
50 J. Go to a 4-year college or university. 
T K. Don't know. 

7. I think ny parents would like me to : 

1 A. Quit school as soon as possible. 

19 B. Finish high school. 

15 C. Go to trade/technical school or junior college. 
62 D. Go to a 4-year college or university. 

T E. Don't know. 
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8. Actually, I will probably : 

1 F. Quit school as soon as possible. 
23 6. Finish high school. 

30 H. Go to trade/technical school or junior college. 
40 0. Go to a 4-year college or university. 
T K. Don't know. 

9. How comfortable do you feel about choosing a future career goal at this point in your life? 
10 A. Very Uncomfortable 

13 B. Uncomfortable 

34 C. Neither Uncomfortable or Comfortable 

J[ D. Comfortable 

T7 E. Very Comfortable 

The following sentences describe some of the ways in diich people might think about themselves. 

Read each of the following sentences carefully and mark the letter on the answer sheet that 
tells how much it is like you . 

Look at the following practice sentence and mark the letter on the answer sheet that tells how 
much you agree or disagree with the sentence. 



PRACTICE 



Strongly Mildly Not Mildly Strongly 
Agree Agree Sure Disagree Disagree 



I am good at art 



A. 



B. 



C. 



•D. 



E. 



If you Choose "Strongly Agree," you're saying that you are very good at art. If you choose 
"Mildly Agree," you're saying that you are OK at art. If you choose "Mildly Disagree," you re 
saying that you are not too good at art. If you choose "Strongly Disagree," you're saying that 
you are very poor at art. 

Strongly Mildly Not Mildly Strongly 





Agree 


Agree 


Sure 


Di sagree 


Disac 


10. I'm popular with kids my own age. 


21 


52 


20 


5 


1 


11. Kids usually follow my ideas. 


12 


£ 


29 


9 


3 


12. Most people are better liked than I am. 


6 


13 


32 


29 


19 


13. It is hard for me to make friends. 


4 


11 


5 


27 


52 


14. I have no real friends. 


3 


4 


4 


10 


79 


15. I'm not doing as well as I'd like to in school. 


36 


32 


5 


14 


12 


16. I am a good reader. 


39 


37 


11 


8 


5 


17. I'm proud of my school work. 


16 


37 


17 


19 


11 


18. I'm good at math. 


22 


33 


14 


17 


14 


19. I'm doing the best work that I can. 


14 


28 


13 


28 


16 


20. I am able to do school work at least as well as 


46 


32 


14 






other students. 


6 


2 
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Strongly 
Agree 

21. tty grades are not good enough. 27 

22. I'm always making mistakes in ny schoolwork. 5 

23. I am a good writer. 21 

Questions About Your School Life 
How much do the following words describe most of the teachers at this school? 







Very 


Pretty 


Some- 


Only A 


Not at 






Much 


Much 


vrtiat 


Little Bit 


All 


24. 


Friendly 


11 


51 


27 


8 


3 


25. 


Helpful 


12 


48 


28 


9 


3 


26. 


Have high hopes for us 


12 


28 


36 


18 


7 


27. 


Talk to us 


18 


39 


27 


12 


3 


28. 


Let us talk to them 


17 


37 


29 


13 


4 


29. 


Care about us 


9 


31 


36 


16 


7 


30. 


Do a good job 


12 


49 


26 


8 


4 



Mildly Not Mildly Strongly 



Agree 


Sure 


Disagree 


Disagree 


32 


8 


18 


15 


16 


16 


40 


23 


. 38 


21 


14 


7 



How much do the following words describe how you feel about most of the students at this 
school ? 





Very 


Pretty 


Some- 


Only A 




Much 


Much 


what 


Little i 


31. Friendly 


13 


51 


28 


7 


32. Helpful 


7 


32 


40 


17 


33. Have high hopes 


8 


28 


43 


16 


34. Smart 


7 


41 


41 


9 


35. Talk to each other 


48 


36 


12 


3 


36. Care about each other 


17 


41 


29 


10 


37. Competitive 


41 


32 


20 


5 



Not at 
All 

2 
4 
4 
2 
1 
3 
2 
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38. The most popular students in this school are: (Choose only one answer) 
48 F. Athletes 
3 6. Smart students 
9 H. Members of student government 
35 J. Good-looking students 
3 K. Wealthy students 

Indicate whether or not you participate in the following activities at school. (Answer yes or 
no for each of the following). 

Yes No 



37 60 
8 88 



39. I participate 1n sports teams/drill team/flags/cheerleadlng. 

40. I participate in student government. 

41. I participate in music, band, drama, or other arts. 

42. I participate in honor society. 

43. I participate 1n school clubs/community service activities. 

Below 1s a list of things which may be problems at this school. How much do you think each is a 
problem at this school? 

Not a Minor Major 

Problem Problem Problem 



17 79 
19 77 
26 71 



44. Student misbehavior (fighting, stealing, gangs, truancy, etc.) 


17 


62 


19 


45. Poor courses or not enough different subjects offered 


40 


40 


17 


46. Prejudice/Racial conflict 


66 


26 


7 


47. Drugs 


16 


49 


34 


48. Alcohol 


18 


45 


36 


49. Poor teachers cr teaching 


33 


48 


17 


50. School too large/classes overcrowded 


59 


31 


9 


51. Teachers don't discipline students. 


57 


34 


8 


52. Poor or not enough buildings, equipment, or materials 


41 


38 


19 


53. The principal and other people in the office who run the school 


32 


34 


32 


54. Poor student attitudes !poor school spirit, don't want to learn) 


23 


49 


26 


55. Too many rules and regulations 


21 


35 


43 


56. How the school is organized (class schedules, not enough time 








for lunch, passing periods, etc.) 


12 


28 


58 
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Issues an d Problems : 

Rpad each one of the following sentences carefully and choose the letter that tells how much you 
agree or dislgree\lith what it says. CHOOSE ONLY ONE LETTER for each sentence. Please raise 
your hand if you have any questions. 





Strongly 
Agree 


Mildly 
Avjree 


Not 
Sure 


Mildly 
Di sagree 


Strongl 
Di sagrc 


57. What I'm learning in school is useful 
for what I will need to know NOW. 


21 


40 


16 


16 


7 


58. What I'm learning in school will be 
useful for what I will need to know 
LATER in life. 


34 


33 


15 


10 


3 


59. I think students of different races or 
colors should go to school together. 


£ 


11 


7 


3 


10 


60. Girls get a better education than boys 
at this school . 


5 


6 


23 


11 


55 


61. There are places at this school where 
I don t go because I m afraid of other 
students. 


6 


8 


6 


10 


68 


62. Boys get a better education than girls 
at this school. 


3 


3 


23 


12 


59 


63. I do not have enough time to do my school 
work. 


15 


28 


13 


26 


18 


64. High school students should have job 
experience as part of their school 
program. 


32 


27 


23 


11 


7 


65. Many stude-'.ts at this school don't 
care about; learning. 


22 


34 


24 


15 


4 


66. Average students don't get enough 
attention at this school. 


17 


29 


29 


17 


6 


67. Some of the things teachers want me to learn 
are just too hard. 12 


91 


1 7 
1/ 


a 




68. Too many students are allowed to 
graduate from this school without 
learning very much. 


19 


22 


23 


17 


16 


69. If I had my choice, I would go to a 
different school. 


11 


8 


21 


18 


42 


70. There are things I want to learn 
about that this school doesn't 
teach . 


29 


24 


18 


15 


13 


71. It's not safe to walk to and from 
school alone. 


5 


8 


11 


15 


60 
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Strongly Mildly Not Mildly Strongly 
Agree Agree Sure Disagree Disagree 

72. I have trouble reading the books and other 



materials in ny classes. 


7 


12 


8 


23 


52 


73. me grades or marks 1 get neip me to learn 
better. 


17 


29 


25 


17 


11 


/4. 1 1 iKe scnooi . 


1Q 


A1 
*H 




19 


19 


75. The grades or marks I get in class have 
nothing to oo witn wiat i reaiiy Know. 


91 


95 
CO 


1Q 


91 
CI 


1 A 


76. I have to learn things without knowing why. 


18 


21 


21 


19 


13 


77. Parents should have a say in vfiat is 
taught at this school. 


17 


27 


23 


16 


16 


78. It is easy for me to get help from a 

counselor when planning ny school program. 


39 


28 


10 


11 


9 


79. We are not given enough freedom in choosing 
our classes. 


27 


21 


11 


23 


18 


80. We are not given enough freedom in 
choosing our teachers. 


49 


19 


8 


12 


10 


81. If I have a personal problem, it would be 
easy for me to get help from a counselor. 


19 


17 


26 


14 


23 


82. If you don't want to go to college, this 
school doesn't think you're very important. 


8 


16 


31 


21 


22 


83. Students should have a say in vrtiat is 
taught at this school . 


37 


32 


14 


9 


6 


84. A person is foolish to keep going to 
school if he/she can get a job. 


4 


4 


9 


16 


65 


85. If I need help planning for a career, it 
would be easy for me to get help from a 
cuunsc i ur . 


00 


9fi 
CO 


1ft 

JLO 


11 

JL JL 


« 

O 


86. I like the way this school looks. 


14 


42 


18 


16 


9 


87. It is easy to get books from the 
school library. 


40 


36 


11 


6 


4 


88. Things in the school library are useful 
to me. 


32 


41 


15 


7 


4 


89. Materials in the Career Guidance Center (CGC) 
are useful to me. 


29 


27 


29 


8 


5 
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Questions About Teaching, Learning & Classroom Work 

All schools teach pretty much the same things, but they may think some things are 
more important than others. . . 

90. Which ONE of these does this school think is the most Important thing for students? (Choose 
only one ) 

7 F. To work well with other people . . . . 

65 G. To learn the basic skills 1n reading, writing, arithmetic, and other subjects 
TJ H. To become a better person 
10 J. To get a good job 

91. If you had to choose only the ONE most important thing for you , which would it be? (Choose 
only one) 

14 A. To work well with other people 

24 B. To learn the basic skills in reading, writing, arithmetic, and other subjects 
32 C. To become a better person 

25 D. To get a good job 

In general, how do you like the following subjects? 



92. English 

93. Mathematics 

94. Social studies (history, geography, 
government, etc.) 

95. Science 

96. Computer Education 

97. The Arts (art, crafts, music, drama, 
dance, creative writing, film- 
making, photography) 

98. Foreign Language 

99. Vocational /Career Education (shop, 
business education, home economics, 
etc.) 

100. Physical Education 



Like 
Very 
Much 


Like 
Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


23 


45 


10 


14 . 


6 


25 


34 


10 


14 


16 


20 


31 


13 


16 


16 


23 


30 


16 


14 


14 


28 


27 


33 


6 


5 


40 


26 


20 


8 


5 


13 


26 


24 


16 


21 


24 


30 


29 


8 


5 


43 


28 


11 


8 


8 
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101. How many hours of homework do you have each day? 

14 A, None 

40 B. About 1 hour 

15 C. About 2-3 hours 
"!> D. About 4-5 hours 

3 E. More than 5 hours 

102. In general , how often do you do your homework? 
21 F. All of the time 

41 G. Most of the time 

21 H. Sometimes 

11 J, Seldom 
3 K. Never 

103. How soon do teachers usually return your work? 

12 A. the next day 
29 B. 2 days later 
S C, 3 days later 
Iff D. 4 days later 

22 E. 5 days later or more 

104. When you make mistakes in your work, how often do teachers tell you how to do it correctly? 
10 F. All of the time 

35 G. Most of the time 
28" H. Only sometimes 

15 J. Seldom 

6 K. Never 

105. How often do your parents or other family members help you with your school work? 

7 A. All of the time 

16 B. Most of the time 
25 C. Only sometimes 
IS D. Seldom 

71 E. Never 

Listed below are four ways students can work in a classroom . Choose the letter on the answer 
sheet that tells how much you like or would like to work in each way, even if you don't do so 
now. 





Like 
Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


106. Alone by myself 


20 


35 


11 


20 


12 


107. With the whole class 


21 


41 


14 


15 


7 


108. With a small group of students, 
who know as much as I do 


39 


35 


12 


6 


5 


109. With a small group of students, 
some w.o know less, some vJio know 
as much, and some who know more 
than I do 


31 


31 


17 


11 


8 
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Listed below are some things that might be used in a class . Choose the letter on the answer 
sheet that tells how much you like or would like to use~each thing, even if you don't use 1t in 
a classroom. 





Like 
Very 
Much 


Like 

Somewhat 


Undecided 


U 1 S 1 1 Ke 

Somewhat 


ui sii Ke 

Very fAjch 


110. Textbooks 


11 


00 




00 




111. Other books 


10 


39 


or 

26 


1 C 

lb 


0 


112. Work sheets 


14 


40 


15 


1 *7 
1/ 


1U 


113. Films, filmstrips, or slides 


A "5 

43 


35 


9 


c 
0 


0 
0 


114. Games or simulations 


39 


29 


16 


7 


A 

4 


115. Newspapers or magazines 


18 


37 


23 


12 


r 

5 


116. Tape recordings or records 


21 


28 


22 


16 


8 


11/. lei ev i s lOn/vioeo 


CA 


01 


D 




1 

A 


118. Calculators 


38 


34 


15 


5 


3 


119. Globes, maps, and charts 


20 


34 


20 


13 


9 


120. Animals and plants 


35 


33 


17 


6 


4 


121. Lab equipment and materials 


37 


30 


14 


9 


5 


122. Computers 


48 


25 


14 


3 


5 


Listed below are some things that you might do in a class, 
sheet that tells how much you like or would like to do each 
cl ass* 


Choose the letter on the 
thing, even if you don't 


answer 
do it in 




Like 
Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Mich 


123. Listen to the teacher 


17 


46 


13 


13 


6 


124. Go on field trips 


60 


23 


8 


3 


2 


125. Do research and write reports, 
stories, or poems 


10 


24 


13 


22 


24 


126. Listen to student reports 

127. Listen to speakers who come to class 


10 

30 


26 
40 


20 
11 


22 
9 


17 
5 


128. Have class discussions 


40 


32 


11 


7 


5 


129. Build or draw things 


29 


28 


18 


12 


8 


130. Do problems or write answers to 
questions 


11 


31 


20 


20 


13 
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Very 
Much 


Like 

Somewhat 


Undecided 


Dislike 
Somewhat 


Dislike 
Very Much 


131. Take tests or quizzes 5 


25 




15 




27 


23 


132. Make films or recordings 24 


23 




29 




11 


8 


133. Act things out 19 


22 




22 




16 


16 


lot* (\cdU lUi 1UII Ui IflLctcbU ou 


1\ 
o± 




13 




8 


5 


135. Read for information 17 


36 




17 




17 


8 


136. Interview people 17 


24 




24 




17 


12 


137. Do projects or experiments that 

are already planned £0 


37 




17 




13 


7 


138. Do projects or experiments that I 

plan 24 


30 




21 




12 


8 


Please indicate how important each of the following items was in your choice 
of classes here at Royal High School. 




Very 
Important 


Important 


Not 
Sure 


Not 
Important 


Very 

Unimportani 


139. Taking classes from teachers I like 


58 


23 




6 


6 


2 


140. Being in the same classes as ny friends 


32 


33 




11 


15 




141. Completing graduation requirements- 


74 


12 




3 


3 


0 

Cm 


142. Learning skills for a future job 


60 


24 




6 


2 


2 


143. Taking classes that will help me be a better 
person 


46 


31 




10 


4 


c 


144. Being challenged by taking hard subjects 


22 


33 




19 


13 


u 


145. Taking classes that will prepare me for the 
future 


55 


26 




8 


2 


2 


146. Getting a wide variety of classes 


34 


32 




17 


j 


2 


147. Preparing for college 


48 


24 




13 


5 


3 


148. Taking classes requiring little work 


10 


18 




29 


31 


15 


149. Avoiding subjects I don't like 


22 


24 




22 


17 


7 


150. Taking classes that are popular 


9 


17 




28 


27 


12 


151. Taking classes ny parent(s) consider 














important 


9 


28 




22 


21 


13 


152. Taking classes where I can get good grades 


22 


30 




18 


17 


5 
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Questions About the Learning Resource Center (LRC) 



153. Have you heard of the Learning Resource Center? 
79 A. yes 

1 B, no 

154. If yes, how often have you gone with your classes to the Learning Resource Center? 
22 F. Never 

52 G. Only once or twice 
TU H. About once or twice a month 
3 J. About once or twice a week 

2 K. Almost every day 

155. How often have you gone to the Learning Resource Center by yourself? 
50 A. Never 

25" B. Only once or twice 
8 C. About once or twice a month 

3 D. About once or twice a week 
3 E. Almost every day 



If you have ever used the Learning Resource Center, have you used any of these services? 
(Answer yes or no for each of the following). 


No 


1 Kfi niannncti'r +ocHnn ■few rparlinn anrl nwth nmhlp(TK 
130. uictgnusv.n. icbtiiiy iur icauiiiy anu niaui yiuuiaiii 


8 


72 


157. Entry testing for proper class placement 


8 


72 


158. Assistance with assignments from classroom teacher 


16 


64 


159. Work on tasks assigned by the Learning Resource Center 


10 


70 


160. After school seminars 


8 


71 


161. Study hall 


17 


64 


162. SAT preparation 


10 


70 


163. Proficiency test preparation 


11 


70 


i64.- Use the computer 


18 


62 


165. Study skills 


19 


61 


166. Language laboratory 


11 


69 


167. Assistance in researching or typing papers 


12 


68 


168. Use the typewriter 


8 


72 


169. Receive individual tutoring 


6 


73 


170. Develop library/research skills 


9 


71 


171. Develop reading skills 


9 


70 


172. Develop writing skills 


10 


69 


173. Develop math skills 


6 


73 


174. Develop listening skills 


12 


68 


175. Develop test taking skills 


14 


65 
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176. Have you received credit for Writing I through the Learning Resource Center? 
4 F. yes 

76 G. no 

177. Have you received credit for Developmental Reading through the Learning Resource Center? 
4 A. yes 

76 B. no 



Strongly 
Agree 

16 



178. The Learning Resource Center is 
helping students at Royal. 

179. Most students know about the resources 
available in the Learning Resource 
Center. 9 

180. I have been helped by the services of 

the Learning Resource Center. 9 

181. I am comfortable about using the services 
of the Learning Resource Center. 11 

182. My work in the Learning Resource Center 
has helped me in my courses. 7 

183. My work in the Learning Resource Center 
has made me feel more secure about my 
ability to do the work assigned by my 
teachers. 6 



Agree 
32 

23 
18 
18 
12 

11 



Strongly 

Undecided Disagree Disagree 



32 



23 
20 
35 
32 

32 



21 
18 
11 
16 

16 



9 

20 
11 
18 

19 



Questions About the Career Magnet School 

Strongly Strongly 

Agree Agree Undecided Disagree Disagree 

184. I understand what the Career Magnet 

School program is trying to do. 20 26 18 11 13 

185. I would like more information about the 

Career Magnet Schools. 28 24 20 6 9 



9 

ERIC 



DO HOT WRITE ON THIS PAGE 



195 



-12- 



APPENDIX B 



Outline of Two-Day 
Work Group Retreat On 
Using and Reporting Information 
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Some Guidelines for June 18-19 
High School/UCLA Workshop 



Thinking About the Use of Information for 
Different Purposes 



• Different contexts (or levels): 

Individual , e.g., student test scores, survey responses, 
ate. used diagnostically on a per student basis. 

Class , e.g., student test scores, survey responses, 
etc. aggregated (e.g., averaged, tabled, etc.) for 
all students in a particular class and used for 
teacher-class planning. 

Department , e.g., student data aggregated for a particular 
department (e.g., math) and used for teacher/administra- 
tor planning. 

School , e.g., data aggregated for entire school (perhaps 
organized by grade, sex or other relevant attribute 
depending on purpose) and used for administrative and 
teacher planning. 



• Let's take a couple of examples: 

Function of schooling questions 
(#90 and #91) 

Educational aspiration questions 
(#6 - 8) 

Instructional grouping questions 
(#106 - 109) 



• Some different ways of looking at survey questions: 
—-One-by-one 

—Contrasting one with another 

— Crosstabulating (detailing the relationship of) one 
with another 
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t Small group brainstorming: 

Think up several examples using survey data (or 
envisioned analyses of these data) of how information 
could be used for: 

— Individual level instruction 
—Classroom le/el planning 
—School level planning 



• Total group brainstorming: 

In particular, in what ways can we increase the potential 
relevency of single items of information by interrelating 
* them with other important information? Specific 
examples are needed . 



• Reporting formats: 

How would you like to see the results reported back? 
—Small work groups take a shot at several examples. 

Are there any general "rules" that emerge? 
— Total group discussion 
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APPENDIX C 



Description of District 
Information System 
File Structure and Contents 
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The district's information system (implemented on a Burroughs Model 
6800 mainframe computer with remote terminal access at district and 
school levels) is made up of a number of files that can be linked 
together by identification pointers (using COBOL) for purposes of 
updating, sorting, merging and selecting information in analysis and 
reporting operations. 

Of essential importance to this project are these five student 

files compiled and maintained by the district: 

CTBS Test Score File * cumulative record of all student test score 
results for all quantitative and verbal scales. 

Activity Record File : Miscellaneous information such as students 
AFDC, GATE, bilingual, and special education statuses; permits 
(smoking, auto, off-campus); extracurricular activities; and 
contacts with health office, principal, counselors. 

Master Record File : Basic linking file consisting of student's 
" arade, sex, address, phone, other personal and family data 
(e.g, mother/father occupation; emergency contacts; health 
data; doctor name); FES, LES, NES, MGM codes; ethnicity codes; 
school entry/leaving date(s); etc. 

Attendance File : Complete record of daily attendance including 
full days and partial days absent. 

Cumulative Grade File : Total number of letter grades of each type 
(A-F), current GPA, total credits class rank, class size, 
expected graduation date. 

Using these (and other) files, basic information on students such 

as class schedules, attendance data, test data, course grades, career 

and educational goal information, records of academic/social referrals 

and conferences, and miscellaneous extracurricular activities and 

activity permits can be stored, accessed, and used for various reporting 

purposes. 

Using terminals (linked to the nuin frame) at the building level, 
administrators, counselors, and other trained staff can access (and 
print) data displays containing the following information: 
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1. Cross-referenced information: more than 175 variables, student 
indicators, and activities can be cross-related by the 
computer. 

2. Current test data and history of test data, 

3. Special education IEP data, 

4. Attendance and re-admits, 

5. Addresses and grids of residence, 

6. Graduation requirements. 

7. Permanent record card. 

8. The four-year educational plan, 

9. Student master record, 

10. Activity-referral form, 

11, Search class by the section number, 

12, Course requests. 

13. Student locator— look up and update. 
14* Master schedule and section number. 

15. Student continuum data (CMI test information), 

16. Proficiency test results. 

17. Career Magnet Schools: paths, programs and course plan, 

18. Budget data. 

19. School -wide academic and activity calendars. 

A more complete description of the district's information system as 
well as a general discussion pertaining to hardware-software issues in 
using such systems at district and building levels is contained in the 
1984 Deliverable by Ender. 
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APPENDIX D 



Anrotated Student- 
and Class-At-A-Glance 
Report Formats 



SECTION; 
TEACHER: 



STUDENT 
NUMBER 

49G43 
49052 
41950 
49274 
42463 
49341 
43686 
49048 
42771 
49050 

43177 
44197 
49286 
49288 
41823 
49291 
49069 
49294 
49295 
49301 

49303 
49306 
49167 
48395 
. 49168 
49309 
49172 
43336 
49129 
42520 

42793 
43660 
43179 
49323 
49109 
4 1481 
44147 
49262 
49329 
43580 



STUDENTS AT-A-GL ANCE 



\ 



STUDENT 
BABE 

AE AMS, WILLIAM B 
ANEEESON, JOHN I 
EABREII, ANDREW ? 
EABFY, LISA S 
EASED, LI II DA C 
CAKCU, JULIE M 
CAMFBELL. BEIAN R 
CAEACCIOLO. MEIAN 
CAEBILLO, D STEVE 
EVANS, MICHELLE A 

f EENEY, TIMOTHY P 
FLETCHER, DASN HI 
GAILEGLY, SHAHN H 
GILGENBERG, MARY 
GECCE, SONIA R 
HARSH* TAMARA I 
HIILENBFANE, STEV 
ROEBAHD, S FRANK 
HULL, ERIC J 
KGVACH, ROBERT S 

LAESEN. TOED H 
LCIZ, DARREN K 
KAC NEAL, CHERYL 
HAESTAS, PATRICK 
MICHIELSEN, PIETE 
CIILE5, PAUL A 
G ECHNELL, 1HEEES 
P BEST A, DAVID BAY 
3CSSELL, DANIEL H 
SAIAS, JO ANN G 

SCHNEIDER, JOSIIN 
SHAS. KEVIN H 
SMITH, LOYAL EAVI 
SOEVINO. GINA MAE 
SEEIIMAN, DUANE A 
SKABE, JONATHAN D 
SYNOEIS, ALLISON 
TIGNAC, LOOIS LEO 
VIEZI, GINA 
KCOD, JOSEPH A 



N 

* B 

A 
D 
E 

12 

12 
12 
12 
12 
12 
12 
12 
11 

12 
12 
12 
12 
12 
12 
12 
12 
12 
12 

12 
12 
12 

12 
12 
12 
12 
12 
12 

12 
12 
12 
12 
12 
12 
12 
12 



C 
M 



C 
H 
0 
0 
L 



6 
3 
3 
3 
4 
6 
6 
6 

6 
6 
1 
2 
6 
7 
4 
4 
1 
1 

6 
4 
5 
3 
4 
6 
3 
2 
2 
6 

6 
5 
8 
5 
4 
5 
6 
1 



E 

H 
X 
P 
E 
C 
T 

4Y 
HS 
HS 
4Y 
2Y 
2Y 
2Y 
2Y 

as 



* D 



A 

B 
S 



12 8 
12 4 



4Y 
4Y 
4Y 
2Y 
2Y 
2Y 
HS 
2Y 
4Y 

2Y 
2Y 
2Y 

4Y 
4Y 
2Y 
2Y 
2Y 
4Y 

4Y 
HS 

21 
HS 
4Y 
2Y 
2Y 
2Y 
2Y 



57 
35 
17 
18 
11 
23 
30 
13 

55 
27 
7 
S 
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CM SCHOOL: CAREER MAGNET SCHOOL. 
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3=EHTEY AND ESSENTIALS 9=BCNT KNCtt ^ 

ED EXPECT : EDUCATIONAL EXPECTATION. -* ^ c \<^ u ' x 

CU=QUIT HIGH SCHOOL HS=F3NISH HIGH SCHOOL 

zY=GO TO TRADE/TECHNIC SCHOOL OR JUNIOR COLLEGE 
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CLASS AT-A-GLANCE 
FALL 84 

SECTION NO: XXXX "7 "j),^^ 

NO. ENROLLED STUDENTS: 35 X**"^ 
NO. STUDENTS TAKING SURVEY: 35 J 



INSTRUCTIONAL GROUPING PREFERENCES 

ALONE XXXXXXXXXXXXXXXXXXXX — *###*#####**# 

WHOLE CLASS XXXXXXXXXXXXXXXXX ************ 

HON SMALL CLASS XXXXXXXXXXXXXXXXXXXXXXXX ###*### 

HET SMALL CLASS XXXXXXXXXXXXXXXXXXXXXX ********** 
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LIKE VERY MUCH 
LIKE SOME 
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DISLIKE SOME 
DISLIKE VERY MUCH 



LIKING OF MATHEMATICS 



************************* 

*********************** 

* 

* 



LISTEN TEACHER 
GO FIELD TRIPS 
DO RESEARCH ETC 
LISTEN STUDENT 
LISTEN SPEAKER 
CLASS DISCUSSION 
BUILD/DRAW THING 
DO PROBLEM/ANSWER 
TAKE TEST/QUIZ 
MAKE FILM/RECORD 
ACT THINGS OUT 
READ FOR FUN 
READ FOR INFO 
INTERVIEW PEOPLE 
DO PROJECT PLNED 
DO PROJECT I PLN 



STUDENT ACTIVITY PREFERENCE 

XXXXXXXXXXXXXXXXXXXX ******** 

XXXXXXXXXXXXXXXXXXXXXXXXXXX ***** 

XXXXXXXXX *********************** 
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xxxxxxxxxxxxxxxxxxxxxxxxxx ***** 

xxxxxxxxxxxxxxxx ********* 

xxxxxxxxxxxxxx ************ 

xxxxxxxxxxx ****************** 

xxxxxxxxxxxxxx ************* 

xxxxxxxxxxx **************** 

xxxxxxxxxxxxxxxxxxx ******* 

xxxxxxxxxxxxxxxxxx *********** 

XXXXXXXXXXXXX — ************ 

xxxxxxxxxxxxxxxxxx ********** 

xxxxxxxxxxxxxxx *********** 
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DISTRIBUTIVE PROCESSING ISSUES 
IN EDUCATION INFORMATION SYSTEMS 

-ft i 1 : o B . Ender 

Th * s report is concerned selected issues ana or obi ems 
i" establishing and operating computerized information 
sys t*' * in local educational systems. Schools and districts 
cenerate and store tremendous amounts of information! Some 
of this information is strictly administrative: personne 1 
records, inventories, purchase orders, etc, while other 
information has implications at the instructional le"el: 

# 

test scores, grades, absences, etc. The utilization of 
information occurs at multiple levels: teacher, counsel sr. 
Dri net pals, district administration, suoer i n tenden ts« an-: 
schccl boards '.Burstein, 1934 i Quran t & Cool e/, 19S4. . 
Until recentlv, the management of information has been 
central zed at the district le»>el '.with tne exception of 
manual filing systems at the classreem and school le*?l>. 
Schools and districts have been slow to take eovantage of 
the advances in information management tnat have taken plr.ee 
in the last dec ade < Ha t h aw av . i ?84 > . 

One remmon comp 1 a • n t , at the school level. ; s that tne 
district data processing cente^ is not responsive to^tne 
school • s need for information. Either, it takes too long to 
get reports from the district or the district does not 
q provide data necessar -- to ma- e aoprocriate educational 
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decisions at :r<e school , class, or student level • One 
solut on is to allow the users of the information. u=?ng 
remote terminals or m \ crocompu ter s « located in the scnools. 
to generate reoor :s and analyze data in a manner most useful 
to their own needs. 

Cne area in which information has been available at the 
school and teacher level is Computer Managed Instruction 
«,CMI). in which items or tests are selected from a test bank 
and progress reports of student status are generated. This 
is one area in which there seems to be some cooperation 
between district level data processing and the needs of 
educational decision makers tDussault, 1?S4; Idstein & 
-!;mal z ♦ 19S3.) . 

The D^rsoect i ve of this reoor t is based on e::er i ences 
gained in working with a high school in a suburb of Los 
rtngeles. For the purposes of this report, the high school 
w:M be referred to as Site ft while the district wsll be 
referred to as tne Mai ley Unified School District. This 
oredomi nantl / middle and lower SES district has data and 
information problems similar to many other school districts 
nationwide. Mai ley Unified has a centralized data 
processing center, whose administrators are ooen to advances 
which will improve information utilization in the district. 
mod end ices I through M describe the Mai ley Unified computer 
system and management information system in more detail. 
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The purpose of this report is to discuss the issues and 
to ermine the z r odI ems that can arise in trying to 
implement school and teacher level access to data and use o-f 
i nformat i on . This report highlights two basic approaches to 
i n -format i on processing: 

1. Centralized data processing — With this approach 
a district uses one mainframe computer system with 
on-1 i ne remote termi nal s in various 1 ocat i ons 
throughout the district. All queries, reoor ts « and 
an al yses are processed on this one compu ter system . 

2. DistriDuted (decentralized) data processing — This 
' aoproach makes use o-f multiple computer systems, 

usually one central main-frame computer system and a 
number of m i cr ocom;iu ter systems located throughout 
the district. These microcomputer systems can be 
jsed as remote terminals to the district main-frame 
or they can De used as stand al one comou ter 
s/s terns « analyzing data indeoe n de n t 1 y o-f the 
district s main-frame. \ see Table 1 / 



Main-frame computer systems and microcomputers -function 



and speec of processing. Mainframe computers often fill 
* arge rooms and have their own air conditioning and 



electrical systems. Mainframes reauire trained operators 



MftlNRAMES us MICROCOMPUTERS 



in basically the same manner, the/ differ in size* capacity. 
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and programmers. They are very expensive to ourcnase and 
cperat? but are cost effective because they can proress 
large amounts of data very qu.ckly. 

Mi crocomou ters, on the other hand, are small desktop 
sized computers that are relatively (compared to mainframe 
computers) inexpensive to own and operate, rtt issue is 
whether microcomputers as part of a decentralised data 
processing system can play a useful role in school and 
teacher level access to information. 

While this section is labeled "Mainframes »*s 
Microcomputer", it is not an either-or 1-ssue. For all but 
very small school districts, central mainframe computers 
will be an important element of both centralized and 
distributed processing systems. In distributed systems the 
orocessino is done by remote microcomputers but the master 
data files are still stored on the central mainframe 
:omou *.er . v 
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DISTRICT vs SCHOOL ISSUES 

E^ery school district office performs s large amount of 
date processing to meet » ts own needs. The kinds of data 
analyses are primarily account mo in nature, since much of 
the information will go to either the state or the federal * 
government or is used in making funding decisions eternal 
to the district. * 

208 



BEST COPY AVAILABLE 

Entf-r 5 



3c hoc 1 5 indicate that thev have a need "for information 
that can be used to he Id r*ake instructional decisions. The 
kinos of data and tvxes of reoorts needed by schools may be 
somewnat different than these used by the district 
admi n i strati on • 

Much of the information collected and stored at the 
district level ttest scores, etc.; could be of use to 
educational decision makers if it were either reported in a 
different manner or aggregated at a different level, say the 
classroom or school level (Burstein, 1?S*4). 

The idea is, that once the data is accessable to the 
schools, tnat teachers and/or administrators will be able to 
sit down at the terminal or microcomputer and ask Questions 
about various aspects of student performance, behaviors, and 
attitudes. In the final analysis, the user (teacher, 
counselor, or administrator) doesn't care whether the 
computer system is an on-line centralized system or an 
off-line distributed system. The user only wants to be able 
to get information ouickly and easily. 



Software ^proorams) that a 1 1 ows users to ask sues t i on* 
aDout information contained within the computer are known 



generics.? lv *s auery systems. Que"'/ systems allow one to 
nose cues- ions such questions as "whicn students ha M 5 GPrtr s 



greater than 3.0 and reading percentiles above ?5?" or 



QUERY SYSTEMS j 



GETTING THE INFORMATION 
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"whi-ri students work more than 10 hours per week, listed 
al phabe t i call y oy teacher." 

On-line query systems are expensive either to c ..r chase 
or to aevelope -from scratch. Many on-line ^uery systems are 
ver-' difficult to use* requiring practically Drogrammer 
level skills to operate. Others may be relatively easy to 
use having a high degree of "user f r i endl yness" . On-line 
auery systems take uo a lot of computing resources. One 
implication of this is that response time to questions can 
become very slew, taking up to half a minute or more. 
Further, the computer center cannot run as many other 
programs at the same time when an on-line auery systems i= 
runn i no. 

One very serious concern with on-line auery systems 
centers around reliability of the system. With an on-line 
sve tern when the mainframe computer malfunctions all the 
users, on their remote terminals. cea=e to operate. A 
small malfunction at the district computer center can 
interfere the <\<ork' of literally hundreds of users. When 
processing is distributed, malfunctions on one microcomputer 
do not interfere with the work taking place on other 
microcomputers. The m i croccmou tgr-based <decen tr al i red) 
s v = terns can also operate independent! ' of the district 
ma i nf r ame compu ter . 

Query s/s terns for microcomputers are neither M erv 
q common nor very easv to use. They require a * ot of 
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comouting Dower -from the micro and sophisticated users 
(Idstein & Hthev. 1984). Mi crocomou ter query systems are 
comparable in difficulty (but not in processing power) to 
current on-line systems that have oeen implemented by some 
di str i cts . 

major difference between on-line query systems and 
microcomputer based systems is one of cost. On-line auery 
systems are much more expensive to develooe or purchase <in 
the tens of thousands of dollars; than microcomputer based 
systems (in the hundreds of dollars). Costs aside, the 
microcomputer solution is often more attractive to users 
because it frees them from dependence on the district 
computing center. 

rtt the same time, because of the power innerent in the 
district's mainframe, it is often easier to generate common 
reports for administrators and teachers on the district 
mainframe computer than to accomplish this on a query 
system. This holds true regardless of whether an on-line or 
an off-line query has Deen implemented. The district 
already has much of the information in their files and a 
staff of programmers that can develop these reports quickly 
and efficiently. 

Having suggested distributed off-line processing and/or 
2uerv systems as one possible solution to getting 
information into the hands of teachers, counselors, and 
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scnool administrators the remainder of this report examines 
tne implications, problems, and issues involved in 
di str i bu 1 1 ve process i ng. 

DISTRIBUTED PROCESSING CONCERNS 

F? 1 e Si ze Issues 

In order -for distributive processing on microcomputers 
to occur, data has to be transferred from files and 
databases on the mainframe computer to the mi crocomou ter . 
One concern in transferring data from district mainframe 
comouters to school m i cr ocompu 1 -r s involves the issue of the 
computers memory capacity. Although microcomputers and 
mainframes work using the same basic principles, the 
mainframes usually have a much greater capacity in terms of 
internal PftM (Random Aczess Memory) memory and external cisK 
memory. It is therefore necessar/ to break laroe files and 
databases into smaller chunks for processing on the 
mi crocomou ters. 

The breaking of files into smaller chunks is most 
easily accomplished by the mainframe itself. Files could be 
extracted for a particular grade level witnin one school or 
even down to the classroom level if necessary. 

Today's microcomputers have internal memories ranging 
from 64K (64.000; bytes to just over half a megabyte. There 
is every indication that the trend *s toward larger interna] 
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memories (Mainframe romputers have much larger memories, 
measured in millions of bytes;. The larger the internal 
memory of the mi crocomou ter the more sophisticated programs 
it can run. Although not always true, one can generally 
count on the tact that larger programs will be easier to use 
and will run -faster . 

External disk memory is an important -factor in the size 
of data files that can be used by the m i crocomou ter . The 
larger the disk memorv capacity the larger the data file 
that can be processed. 

External disk memories for mi crocomou ters can be of two 
types: Floppy disks or hard disk. Floppy disks have 
capacities of 140K to one megabyte, ftgain the trend is 
towards larger capacity floppy disks. Floppy disks have the 
advantage that with dual floppies, it is quite easy to make 
backup copies of important data on a regular basis. The 
main drawback is that floppy disk capacity is too =mal 1 for 
many use tu 1 data storage needs . 

The hard disk ha^e much larger capaci-y than flooov 
disks. Hard disks typically run from 10M < 10 1 000.000; bytes 
to 40M, 60M, and even 100M. These kinds of capacities are 
certainly sufficient for school level databases and files. 
Tne drawbacks to hard oi sks are the i r on ces, expensi ve 
<S2,5Q0 to $15,000); and the difficulty in backing up such 
large amcunts of data on a regular basis. 
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Backing-Up Data 

» 

It should be remembered that mainframe computers 
maintain a staff of proorammers and operators, one of whose 
important tasks is to make regular backups ot" the data 
■f»les. District computer centers do this on a daily basis. 
Every data file that is used during the course of the day is 
copied, usually onto magnetic tape, and saved. If anything 
should happed to the original file then the backup can be 
used to rep 1 ace i t . 

Data files can be lost in a number of ways. They can 
be accidently erased, there can be a hardware failure in the 

4 

disk system, or a power failure can occur while a data file 
is being used. These Kinds of occurances are not that 
uncommon. It is very important to have backups of data 
f lies. 

Off-line processing using microcomputers with floppy cr 
hard disks need to develop policy concerning the backup of 
moor t ant files and prcgrams. 

Comoat [bjJJ ty I ssues 

attempts to develop distributed processing systems can 
run into several problems concerning compatibility of data 
files. Off-line processing implies that data files or 
sub-sets of data files be transferred from the district 
mainframe computer to local microcomputers in schools. If 
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tner e are compatibility problems then these tile transfers 
cannot take place, 

C'ata Formats 

Main-frames and microcomputers can have character 
formats which are incompatible with one another. 
Microcomputers typically use ASCII (American Standards Code 
■for Information Interchange) representation -for the 
printable characters and numbers. Many mainframes also use 
rtSCII, but some may use other conventions* such as EBCDIC 
(Extended Binary Coded Decimal Interchange Code), found on 
many IBM mainframes. Tne software which transfer data files 
from the mainframe computer to the microcomputer must be 
able to translate from one representation to anotner. 

Even when two computers can transfer data, it is not 
necessarilv the case that the files transfered from the 
mainframe to the microcomputer can be read and used by t^e 
microcomputer. Files themselves have certain structures. 
p i ! e = from a mainframe computer may have a structure that s 
incompatible with file structures allowed on the 
mi crocomou ter . 

The wavs in which numbers are stored can be very 
different between different types of computers. Some 
computer systems store numbers as packed decimals (in >*hicft 
every two digits of a number are represented in one byte, 1 
while others may store numbers as any one of a dozen 
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different binary representations. Unless the numbers * an De 
conuerted -for one data repi - mentation to anotner, tiles 
cannot be effective! transferred from mainframe to 
m i cr ocoit tuter. 

Language Compatf?- 1 i ty 

Whether we are talking about insurance compan * - ~. or 
school districts, COBOL is the predominant data processing 
language. COBtt: has built into it a number of -features that 
m?Me the use *nd updating o-f large -files easy and practical. 
T he major problem that occurs when one tries to use -files 
created toy COBOL with other, non-COBOL, programs is that the 
■files are not necessarily compatibile. That is, it is. 
possible -for a COBOL program to create -files that onlv a 
spec i -f i c COBOL program can ac* ?ss. Although it is possible 
to use COBOL on some microcomputers, it is not necessarilv 
compatible with the COBOL that is found on main-frame 
ccmpu ter s . 

T *-pical data -files -for major statistical pacKaoes. such 
3s SPSS, SAS . and BMDP make use o-f sequential te.-. t -files, 
Seauentia! te:;t files store information contiguous!' in one 
area on the disk. For example, to obtain information on the 
tv«en?v— f i f th student in a file, you would haue to read 

through a"! the i n -forma t i on on the first twenty-four 4 
students. Many microcomputer statistical programs ana 
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cS-at3.ba.se systems also make use of sequential tries, a! thougn 
equally manv u = s their own unique structures. 

COBOL -files, on the other hand , ma i n t ai n un i Que 
structures that can involve the use of random access -files. 
That is COBOL allows -for the use of pointers that can go 
from one location in a -file to another location without 
having to read all the information in between. 

To use the COBOL -files with an off-line distributive 
processing system reauires a program that can read the COBOL 
■files and translate them into sequential text -files. This 
is most easily accomplished i -f the district's main-frame 
c-'uld merge the necessary -files and generate the appropriate 
sequential text -file. 

Multiple Files 

Not only does COBOL allow -for the use o-f pointers 
within a -file but it also allows -for the simultaneous use of 
ffiuitipie -files. District data process i ng systems working in 
COBOL typically split their databases up into man*' sub-files* 
Thus, in COBOL based systems, it is typical -for information 
needed in a single report to be -found in many different 
files. For instance, there can be a Student Master File* an 
attendance File, a Class Poster FMe, and so on. 

Distributive processing systems may not allow for more 
than one or trio files to be in use at one time. When 
workinQ '.-.nth ot-f-line mi crocomou ter-b&sed systems, it may 
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become necessary to merge the reouired files together into a 
s i ngl e file. 

There are seme mi crocomou ter programs that aP~'» the 
use of multiple files, Dbase II »*and the brand new Dba=e 
1 11^ is a mi crocompu tei database management program which 
allows several related files open at one time. In this 
respect it is similar to COBOL; however, the Dbase files 
themselves directly are not compatible »gi th COBOL files. 
Another, very popular database system, DB Master makes use 
of it = own structure. 

Although it is more efficient during normal data 
processing on the mainframe to make use of COBOL' s unique 
file structures and multiple files, it is usually possible 
to write a program that will combine (merge* the necessary 
COBOL files into one seouential text file for u=e on a 
mi cr ocomou ter . 

rtrch i vaj Files 

-toother difficulty in analyzing district mainframe 
files at the school level on microcomputers is that the 
files may be stored as archival files. That is, the file= 
could contain information on the students for every year o~ 
semester that thev were enrolled. Thus records for a given 
student may app *r many times within the same file. This 
creates no problem for the mass.- am* yst*-s sine* the:. 
COBOL program uses pointers that quickly locate the most 
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current record -for any given student. However « in working 
with data on off-line micr«n router systems, there can be a 
lot of con-fusion about which entry i correct one -for 

an/ given student. Further* archival -files are usually too 
large -for mi crocomDu ters to handle. There-fore, it is 
necessary when creating files -for use on microcomputers to 
download transfer) only the most current or active record 
-for each student. 

Commun i cat i on I ssues 

Many o-f the issues looked at so far are concerned with 
the use o-f -files aerived from mainframe computer systems. 
Nothing has been said *bout the process of moving the 
information from mainframe to microcomputer. 

Communication-, in this context, refers to the communication 
between mainframe computer and -oi crocomputer . 

Computer . can be hooked-up together in two ways: I) 
direct cor. ^ct and 2) dial-up. Direct connect means that 
the two computers are physically wired tcgeth*. * Dial--;;* 
svstems make use of a telephone hook-up between computers. 
In order to make use of a dial -up arrangement each computer 
must be ecu toed with a modem (Modulator/Demodulator). 
Modems are rated in terms of how fast they transmit data, 
300 SftUD modems transmit about 30 character per second while 
i200 EmUD modems can transmit about 120 cos. 1200 Baud 
mooems are the newer technology and conseouentlv cost more. 
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ttt tho microcomputer end 300 BAUD modems run between $100 
and $250, with 1200 BAUD modems in the $400 to $600 ranci- . 
On the main-frame side of the hook-up modems are much more 
expensive, in the thousand* f dollars. Because of the cost 
of charjmg over, many district computing systems still 
ooerate at 300 BftUD. 

On the face of it, communications between mainframe 
computers and microcomputers seems to be very straight 
forward. B^th the mainframe and the microcomputer require a 
modem set to the same baud rate. However, very little 
involving the use of computers is strictly a Hardware issue. 
In order to have effective communications, each system 
requires the appropriate communications software. In 
addition to transferring data from one system to another, 
the communications software may have to translate data so 
that it will be compatible for each -.vstem. 

Typically this is done through the use of a 
communications package in the microcomputer. This 
communications Dackage allows ths- micro to operate as a. 
terminal to the mainframe, that is, the microcomputer 
behaves as if it were a remote terminal connected to the 
mamframe. It also allows the microcomputer to download 
ftles •> transfer files from Mainframe to mi cr ocomou ter> and 
uoload files < transfer files frcm the mi crocomou ter to the 
mainframe). The mainframe also needs to have appropriate 
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software that allows tor communication with m i crocomou ter 
acting as remote terminals. 

In university based systems communications between the 
mainframe and mi crocomputer = can take place in one of two 

':)ays : 

1> The mi crocomou ter communicates directly with the 

ma i n -frame' s on-1 i ne i n ter act i ve edi tor . Or 
2) the mi crocomou ter communicates through a -file 
trans-fer program that directly transfers -files 
■from system to system. 
Either of these systems is usually satisfactory. The 
choice usually comes down to whether the user is just going 
to transfer files or will be running programs and viewing 
the results on the microcomputer. 

Universities tend soend a lot of time and money to make 
communications easy and efficient. School districts, 
typically, do not have the same resources to develop such 
sophisticated communications systems. The idea of hooking 
u ni crocomou ters to the district's main computer is 
relatively new so many districts have not as yet dealt with 
the many problems involved in communications. 

Gr ash i cs 

Graohic displays are one area in wnich ni crocompu ter s 
seem t « have an advantage over the mainframe, especially 
da i ■* Processing mainframes. There is a lot of very good 
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graphics so-ftw*. • available -for microcomputers. By and 
large, the microcomputer <jraph:*s sot' ^re is less costly 
and easier to use than the maon-frame variety. It is 
certainly possible to download -files -from the district 
ma i n -frame to the school mi crocompu ter and produce h i gh 
duality graphs and charts. 

There are some problems with graphic display o-f data 
that have to be anticipated. For one, there is the issue o-f 
the data -format and/or -file structure. While both these 
issues may have been dealt with at the mi crocomou ter level, 
the graphics program may require yet another -format or -file 
structure. Very o-f ten graphics programs have special 
•formats that oreclude the use o-f general sequential text 
•files. Some graphics programs want only the data -for tne 
graph being plotted and not all the other data associated 
with it. Thus, data -for a graph o-f GPA versus hours worked 
would have to be extracted -from the larger database -file. 

another potential problem is concerned with the 
production o-f large numbers o-f graphs and charts, say one 
•for every class. The problem here lies in the -fact that 
most graph i c s so-f twar e is de s i gned to produce custom gr aphs 
or charts and is not designed for batch production of 
hundreds o-f the same graphs and charts are produced on 
slightly' di-f-ferent sets o-f data. 

Multiplicity Qj Databases 
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One problem that can occur when users download Darts of 
larger databases, -for local use on microcomputers, i= that 
multiple nonequ i v al en t versions of the database can be 
■formed. This comes about because local users update their 
local database with different information that does not -find 
it= way into the centralized database. In a short time, it 
is possible to get different answers to the same question 
depending upon which computer and which database is Queried 
». Idstein & Athex, l'?S4> . 

Code books 

* 

Another area where one should exercise caution is in 
terms of the internal data codes that are used by district 
data processing programs. Sometimes information may be 
stored as actual values, at other times information is coded 
and abbreviated. Often only the system programmers have 
access to the codes and abbreviations that are used. This 
13 because users of the mainframe system use on 1 v the 
programs written by the district and do not use the files 
themse 1 ves. 

In research settings, codebooks are commonly produced 
and distributed to all workers on a project. Such code book = 
aren't always easily available for district mainframe data 
files. A considerable amount of time may be spent working 
with the district data processing people, developing a 
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The Idstein & Athey <1?34.' and the Murray <1?S4,« papers 
Mst a number of other problems that can occur in 
implementing educational information systems. For example: 
audit trails lost; the tendency to want to collect and store 
everything; more errors in data entry, due to more points in 
the system where errors can occur; redundant reports; 
incomplete and poorly organized reports; need -for more 
training at ^11 levels; more hardware problems, due to more 
hardware; need -for on-line help -facilities; need tor a user 
suopor t sery i ce . 

Using microcomputers in the schools to assist in mak'ing 
educational decisions is a relatively new undertaking. 
There are still many problems to be identified and solved 
before their use becomes a common practice. Microcomputers 
and their software are becoming more powerful and 
sophisticated everyday, and what may be difficult now, will 
surely be easier in the future. 
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Tobte I 

Comparing Centralized /'^Decentralized Computing 



Centralized Computing Decentralized Computing 

System 



Type of computer(s): 


One mainframe computer. 


Many microcomputers. 


Type of processing: 


On-line processing 


Off-line processing. 


Processing done on: 


Remote terminate. 


Remote computers. 


Size: 


Large, roomeissi. 


Small, desktop sized. 


Computer malfunction: 


All terminate dovn. 


Only one computer down. 


Processing speed: 


High speed. 


Slow speed. 


Response leg: 


Can be very slov. 


Very fast. 


Internsl memory: 


1M + 


64K to 512K 


External memory: 


1Q0M + 


140K to 6QM 


Operators: 


Highly trained. 


Not much training. 


Operating cost: 


Very expensive. 


Moderately expensive. 
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Appendix I 



Uallev Unified School District 
Comou ter Cen ter 



Computer Mam-frame: Burroughs B6300 purchased in 1973. 
Main Memory: 1.2 million bytes. 

Disk Memory: 1.2 billion bytes. 

Other per i pher i al s : 2 tape drives, 2 printers, a nark sense 

scanner , 120 term i rial s. 

Twenty to 25/< of the computer time devoted to district 
accounting needs. Approximately 75% of the computer time 
devoted to student administration needs. 

One hundred thousand to 120,000 pages of reports per month. 
Thirty-five percent of the reports are special reauests from 
the schools or the district office. 
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Valley Unified School District 
Educat i nnal Dart a Process i ng Serv ices 



In 1 ?o? . tne Valley Unified School District embarked upon a 
long-range plan to automate many of its information 
processing meeds. This appendix contains a summary of the 
current status of their system. 



PHILOSOPHY AND OBJECTIVES 



It To control growth of administrative and clerical costs. 

2. To produce in a meaningful and timely manner all 
operating and comparative information needed for effeicient 
di str i c t opera t i on . 

3. To provide information for evaluating educational 
programs based on student achievement. 

4. To provide a Vocational Program in Data Processing. 

5. To make computer time available to both teachers and 
studen ts . 



To eliminate duplication of effort by maintaining one 
central file on all information which meets both internal 
and external needs of the district. 

7. To eliminate known inefficiencies that ari^e from 
inadequate, slow, untimely, or unavilable information. 

To orovide data processing services to additional school 
di str i c ts. 



BASIC SYSTEM FUNCTIONS 



The following is a description of the files that are used in 
the basic system functions of the Educational Data 
Processing Services. These files and the description of 
thier use gives a good overall picture of how the system 
functions and how the various Pieces fit together. 

STUDENT MASTER FILE 



ERIC 



The Student Master File is the key to the entire system. 
Almost every program within the system relies on the Student 
Master File for various forms of information. This file is 
updated on a daily basis from remote terminals. As a 
protective feature, updating can only be accomplished from a 
terminal in the student's own school. 
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STUDENT SCHEDULING 

The schedulino package handles a variety of methods in 
scheduling students such as traditional* flexible, and 
modular scheduling. The schedule must be able to accomodate 
a -four quarter system with up to 20 periods a day. The 
system is generalized to the point of allowing each school 
to have a totally different method of scheduling classes and 
ac 1 1 m i ties. 

PERMANENT RECORD 

The permanent record contains both grades, credits, and test 
scores. The main source of input for this file is from the 
grade reporting system and the yearly test scoring 
procedure . 

ATTENDANCE ACCOUNTING 

The attendance accounting system services the average daily 
attendance requirements and it is also used as a counseling 
tool. Prior to installing remote terminals, it was not 
feasible to autimate period attendance, as is currently 
done . * 

PROFICIENCY TESTING 

The proficiency testing system is designed to measure hew 
proficient a student is in a given discipline. To 
accomplish this task, the various disciplines of reading, 
language, and math are divided into subskills for testing. 

STANDARDIZED & TEACHER MADE TEST SCORING 

Test scoring information is gathered through the use of 
marked documents which are produced from both the student 
master file and student schedule files. The system reads in 
the mark sense documents, convert the responses to raw 
scores based on a key, and convert the raw scores to 
converted scores based on computation or table look up. 

INDIVIDUAL EDUCATION PLAN (I.E. P.) 

This system manages data for handicapped children's 
education program. Parents work with school personnel, 
joint! y makino decisions regarding the ch i 1 ds needs. The 
computerized I.E. P. is used as a management tool to insure 
that each handicapped child is provided special education 
and related services. 

COMPUTER MANAGED INSTRUCTION '.CM I) 
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This system keeps track o-f student progress/pro* i c i enc i es in 
basic skill areas. Currently it is being utilized in Matn, 
Reading, and Language. 

INSTRUCTIONAL MEDIA 

Thr f s system is based on a library cataloging application 
\th i ch is mai ntai ned by remote termi nal s. The -files contain 
media titles, subjects, authors, annotations, and 
publishers. Media catalog, indexes, and shel-f lists among 
othe things are also available -from this system, 

ADULT EDUCATION 

Tne Adult Education System is used -for maintaining a 
student's Master Record, Schedule o-f Classes, grades, 
credits, and Pro-ficiency Test Scores, Data is entered into 
the system by optical scanned -forms and/or terminals. 

PERSONNEL RECORDS 

Personnel accounting -functions are totally integrated into a 
computerized filing system containing most o-f the 
operational data required by the district. This -file 
contains such things as name, address, phone, emergency 
data, credentials, employment history, skills inventory, 
-fringe bene-fits, etc. 

BUDGET PREPARATION 

The budgeting system is the basis for the entire 
appropriation ledger accounting system. The budget -files 
contain in-formation which re-flects almost every conceivable 
major item being-budgeted -for in the oncoming /ear. 

APPROPRIATION ACCOUNTING 

The appropriation accounting system is an up to date history 
o-f all encumbrances, expenditures, trans-fers, abatements, 
and other sorts oi transactions. This -file is maintained 
through many sources such as the receiving department, 
accounts payable, purchasing department, etc. 

ACCOUNTS PAYABLE 

The system has the capability oi receiving invoice d £, t a , 
comparing it ot the purchase order data, ana kicking cut the 
transaction i* all things are not eaua! . It invoices match, 
previous prices and receivables, the system will make an 
entry into the appropriate ledger tile. 

PUPCHhSE ORDER GENERATION 
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Tne system or oduces our cha.se orders using a standardized 
catalog which reflects almost e»»erv item that the district 
purchases during the year. 

FURNITURE , EQUIPMENT, & SUPPLt CATALOGS 

The primary -function of this -file :s to supply information 
•for generating purchase orders, keening track o-f amounts in 
order, /ear to date amounts received, printing out various 
catalogs, and keeping track o-f warehouse inventory reorder 
Quan titles, e tc . 

INVENTORY CONTROL 

Through the use o-f an automated system, operational supplies 
are mai ntai ned on a constant basi s in the district 
warehouse. Terminals placed in every department and school 
initiate the automatic handling o-f warehouse supply 
reau i s i 1 1 ons . 

PROPERTY MANAGEMENT 

The system is used to maintain an* ongoing inventory -file o-f 
all equipment items which are valued over twenty-five 
dollars. This file contains item description, acquistion 
date, acquisition price, serial number, purchase order 
number, amount purchased, and physical location within the 
district. 

PROPERTY MAINTENANCE 

Through the use of an automated system, work orders are 
received by Maintenance and Facilities. As work is 
warrented, requests are given to the designated personnel in 
each School/Department and a work order is initiated via the 
term i nal s . 

CRIME REPORTING 

T he Crime Reporting System maintains a detailed record of 
Crime/Vandalism within the district and a summar/ of cost 
breakdowns and types of crimes. 
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Valley Unified School District 
Standard Reports 



The -following is a list of the standard reoorts which are 
generated by the Educational Data Processing Services. The 
report titles are classified by the basic system function 
they serve, 

STUDENT MASTER FILE 

Teacher Information Sheet 

District Student Address Directory 

District Master Address Report 

Student Living Attendance 

Locker Assignement Loo 

Self Scheduling Course Request 

STUDENT SCHEDULING 

Students Listed by Course Priorities 

Course Master List 

Course DEpartment List 

Master Schedule Exception Report 

Master Schedule Proof List 

Course Reauest Listing 

Potential Conflict Matrix 

Section Enter/Drop List 

Act i v i ty Cal endar 

Student Graduation Status 

Grade Reporting Exceptions 

Drop* Fail and Unsatisfactory List 

Principals Recognition List 

Grade Proof Report 

Grade Proof List 

Grade Distribution By Subject 

Grade Distribution By Teacher 

Progress Report Exception List 

Toacher Progress Report Proof List 

Sports Ineligibility Report 

PERMANENT RECORD 
Perminant Grade Record 

ATTENDANCE ACCOUNTING 

Students Living In Attendance Boundry 

Mai id Locator Count 

Uncleared Absent Report 

Weekly Attendance Reoort 

Unexcused Absence Report 

Bimonthly Freauent Absence Report 

Attendance Exception Report 

Attendance Detail Report 



ERIC 




BEST COPY MAW-ABLE 



Principals Attendance Report 
Monthly Attendance Report 
Attendance Record For AIT 
Enrollment By Grade And Sex 
Enrollment Summary 
Summary By Special Classes 
Monthly Enrollment Report 
Weighted Class Sire Report 
Trend For Weighted Classes 

PROFICIENCY TESTING 

Rasch Testing Values 

Proficiency Testing Detail Report 

Test SCore Distribution 

Pro-ficiency Testing Results 

Proficiency Testing Status 

Pro-ficiency Response Report By Test 

STANDARD 1 2 ED & TEACHER MADE TEST SCORING 

Physical Performance Test 

Teacher Instructional Strategy 

Student Detail Report For Group Testing 

Class Summary Report For Group Testing 

School Summary Report For Group Testing 

District Summary Report For Group Testing 

Simple Response Tally 

Title I List By Teacher 

Title I List By Grade 

Individual Studet Quartile Ranking 

School Quartile Summary By Grade 

Students Below 37th Percentile in Reading and/or Math 
Scores And Wrong Response Record 
Frequency Analysis 

Frequency Distribution With Percentiles 
Standford Early School Achievement Test 

INDIVIDUAL EDUCATION PLAN <I.E,P«) 
I.E. P. For The School Year 
Ongoi rig Indi v i dual Educat i on PI an 
Drivers Ed List 

Special Education Classification List 

Special Education Pupil Count 

Undup I i cated Age Classification Count 

Ethnic, Sex, LES-NES Classification Matrix 

Special Services Tally 

Student Flow 

New Student Classification Ethnic Code Matrix 
Regular Program Participation Tally 
Adaptive PE List 

COMPUTER MANAGED INSTRUCTION <CMI> 

Individual Student Profile Of Incompleted Tasks 

Tasks Yet to Be Completed 

Continuum Status Report 
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I NSTRUCT I ONAL MED I H 

Media Center Audi o-Ui su al Department List 

Library Maintenance 

Library Media Additions 

Media Inventory By Dewey Prefix 

ADULT EDUCATION 
Teacher Sequence 

PERSONNEL RECORDS 

Follow Ud List For Rehire Letters 
Certificated Employees 
Posi t i on Codes 
Location Codes 
Subject Codes 

CI assi f i cat i on Salary Schedule 

CI assi f i cat i on Management Salary Schedule 

Per sonne 1 Pos i t i on/Locat i on Repor t 

Employee Master Record 

School District Bargaining List 

Personnel Inf ormat i on 

Quarterly Personnel Payment Record 

Payrol 1 Listing 

Processing Transactions 

Employee Retro Report 

Payroll Balance Listing 

Credential Report 

I nsur ance Carr i ers 

Employee Insurance Categories 

Employee Credential And Education Record 

Insurance Analysis Report 

Monthly Insurance Report 

Employee Insurance Record 

Employee Probation Report 

Applications Status Report 

CI ass i f i cat i on Sal ar i es 

Quarterly Personnel Payment Record 

Certificated Payroll Information 

Not i f i cat i on Of Salary Placement 

BUDGET PREPARATION 
Buget Re auest Form 
Balance Buget Input 
Buget Balance Totals 
Reauested Buget 
Responsibility Detail 
Obiect Detai 1 
Ob.iect Summary 
Preliminary Buget 
Ten tat i Me Buget 
Progr am De t ai 1 
Pr ogr am Summar y 
Program Detail By Object 
Publ i shed Buget 



***** 



State Program Summary 
Final Buget 

Current Cost ot Education Per Average Daily Attendance 
Responsibility Summary 
State Program Detail 

APPROPRIATION ACCOUNTING 
Regular Board Meeting Agenda 
Accounts Appropriation Ledger 
Cos t Distribution 

Aopr opr i at i on Expenditure Trans-fer Report 
Apsropr i at i on Ledger 
Level Summary 

Payroll Expenditure Error Report 

Di sencumber Total 

Ledger Update Error Report 

ACCOUNTS PAYABLE 

Apprpriation Ledger Direct Entry Proof Totals 
I nval i d I nvoi ces 
Current Year Expended 
Invoicer Report 

PURCHASE ORDER GENERATION 
Purchase Order 
Balance Outside Purchases 
Invalid Requisition Report 
Purchase Order Follow-Up Report 
Invoice Not Received Report 
Inventory Status Report 
Receving Report 

FURNITURE < EQUIPMENT, SUPPLY' CATALOGS 

General Purpose Catalog 

Warehouse Picking Report 

Subscr i d t i on Catal oo 

Authorize Bid Acceptance 

Vendor Organization Plan 

Bid Form 

INVENTORY CONTROL 

Monthly Statement ot" Warehouse Charges 
Warehouse Requistion Activity 
Warehouse Transaction Report 
Back-ordered Warehouse Requisition 
Back-ordered I terns 
Inventory Status Report 
Warehouse Inventory Ver i -f i cat i on 

PROPERTY MANAGEMENT 
Fi xed Data By Noun 
Fixed Data By Stock Sequence 
Noun Listing 
Verb Li st i no 

O 
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Manu-f ac ture Listing 

Serial Master Maintenance 

District Serial Master File 

Furniture and Equipment Inventory 

Furniture and Equioment Inventory Maintenance. 

PROPERTY MAINTENANCE 
Workorder Status Report 

CRIME REPORTING 
Crime Report 

Crime And/Or Damage Reporting 
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Appendix IV 

Mai ley Uni-fied School District 
Sample o-f Non-standard Reoort Requests 

Hany reports and printouts generated by the district 
computing center are derived -from user requests. These 
requests may be submitted in writting or they may be direct 
requests -for reports -from remote terminals. These 
non-standard reDorts and printouts may be generated only 
once or at most a -few times. I-f there is su-f-f i c i ent need 
•for a non-standard report, it can then be incorporated into 
the Basic System Functions. The -following is a sample o-f 
some o-f the non-standard reports that have been requested by 
the schools and district o-f-fice. 



ACCOUNTING AND BUSINESS FUNCTIONS 
LIST OF ACCOUNTS BY PROGRAM ic DISTRIBUTION 
LIST OF DATA COM PROGRAM WITH DA/TIME C0DE=2 
ALL DATA FILES USED IN REPORT GENERATOR 
GENERAL FUND 4000 ACCOUNT SUMMARY 
PROGRAM LIST OF ALL BUSINESS PROGRAMS 
ACCOUNTS WITH NO ACTIVITY FOR 3 YEARS 
OPEN PURCHASE ORDERS 

DATA COM TRANSCODE WITH THEIR PROGRAMS 

REPORT WORDS USED FOR GENERATING REPORTS 

LIST OF ACCOUNTS BY PROGRAM AND DISTRIBUTION 

GRID ANALYSIS USED OTHER RINFOR SPECS TO CREATE DATA 

PERSONNEL REPORTING FUNCTIONS 
LIST OF ALL ACTIVE APPLICANTS 

LIST OF I-.MPLOYEES WITH TOTAL MONTHS OF SERVICE SORTED 

STUDENT REPORTING FUNCTIONS 
LIST OF STUDENTS IN CAREER MAGNET SCHOOLS- 
IMMUNIZATION REPORT FOR LAST YEARS STUDENTS 
MILITARY LIST BY SCHOOL AND GRADE 
ROOM LIST WITH PARENTS NAME AND PHONE 
LIST OF ALL STUDENTS IN WORK B<PERIENCE CLASSES 
STUDENT ABSENCES WITH LUNCH PASS 
TO THE FAMILY AT: 

ACTIVITIES OFFICE ATHLETIC FEE LIST 
I1TH GRADE STUDENTS 
YEARBOOK LIST 

STUDENT POPULATION BY SCHiWL 
HISPANIC STUDENTS WITH GPA>3.0 

LIST OF FALL REFERRALS COMPARED TO SPRING UL' iiRRALS 

TO THE PARENT'S OF 

CAREER MAGNET SCHOOL ZERO LIST 

LIST OF STUDENTS IN A. P. CLASSES 

?TH GRADE STUDENTS NUT ENROLLED IN MAGNET SCHOOLS 
ESL BILINGUAL STUDENTS 
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LAST KNOWN ADDRESSES OF CLASS OF 7? 

LIST OF FIRST REFERRAL DATA FOR APPLICANTS 

LIST OF ALL STUDENTS WITH RIGHTS OF IS YEAR OLD 

SPECIAL ED STUDENTS 

LIST OF ALL BOYS WITH GRADE/COUNT 

LIST OF ALL GIRLS WITH GRADE/COUNT 

LIST OF ALL STUDENTS WITH CREDITS EARNED 50 OR LESS 
LIST OF ALL STUDENTS INTEACHER AID CLASS 
CMS NON-ENROLLEES 

LIST OF FIRST REFERRALS ON YES SYSTEM 
PERCENTILE > 95 FOR 8TH AND ?TH GRADERS 
13 YEAR OLD STUDENTS WITH ATTENDANCE PROBLEMS 
ALPHA HOME ROOM LIST 

HISPANIC STUDENTS WITH GPAM.9 WITH ATTENDANCE PROBLEMS 
ATHLETIC PARTICIPATION AND ATTENDANCE 
TOTAL READING PERCENTILES 
STUDENT BIRTHDATES 
SPECIAL TEST 

MASTER LIST OF STUDENTS IN CMS 
10/11 GRADERS WITH GPA>3.0 FOR CONTENTION 
10/12 GRADERS WITH GPA>3.6 FOR DODGER TICKETS 
STUDENTS THAT ARE ON FREE LUNCH 

LIST OF ALL STUDENT CREDITS EARNED OF 190 OR LESS 

ALPHA LIST BY TEACHER 

GPA LIST BY SCHOOL AND GRADE 

3RD PERIOD LIST WITH STUDENTS AND BIRTHDATE 
LABELS BY GRADE OF HONOR ROLL STUDENTS>2 . 99 
MASTER LIST OF STUDENTS WITH GPA>3.2 AND PERCENTILE 
CTBS BATTERY FILE 

LIST OF ALL APPLICANTS IN YOUTH EMPLOYMENT SYSTEM 
12TH GRADERS WITH GPA>2.9? 

ALPHA LIST WITH NAME GRADE AND PHONE NUMBER 
LIST OF STUDENTS SORTED BY TOTAL CREDITS EARNED 
LIST OF STUDENTS PERCENTILE WITH LESS THAN 51% 
SUSPENSION REPORTASB CARDS AND YEARBOOK PURCHASERS 
ASB/YEARBOOK PURCHASES BY GRADE & GPA 



MISC 

LIST OF CHAPTER I CHANGES 
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Si te A Hi gh School 
Computerized Acoun tab i 1 i ty -for Student Achievement 

The original design o*f the Computerized Accountability -for 
Student Achievement (CASA) Project proposed to correct the 
problems that counseling staffs at the secondary school 
level have with managing information effectively. The 
necessity of increased efficiency in managing information 
was the driving force behind CASA. 



1. The Counseling and Guidance Service is one of the most 
valuable resources available to the students. 

2. The counseling program has not been a program for change 
or adjustment to student needs. Little accountability has 
been required from counseling programs in the past. 

3. Counselors, administrators, and teachers are not privy 
to accurate, up-to-date, specific set of information 
indicators which could lead to changes in the understanding 
of student needs and to concrete changes in student 

curr i cu 1 ar pr i or i t i es . 

4. The frustration of parents is often elicited by 
conflicting and inadequate reports of student progress. 
Students also feel a similar frustration when placed in a 
Position of inadequacy and dependence on a potentially 
mis-informed authority which lacks accurate information. 



1. Provide for the accountability of counseling and 
gu i dance serv i ces . 

2. Provide varieties of relationships between student 
variables and trends associated with programs. 

3. Provide information for staff deve!ooffient ? curricular 
design and program development. 

4. Capability of analysis of relationships and trends 
concerning critical student problem areas such as truancy, 
dropouts, substance abuse, etc. 

5 . Produce students' manes through an anal ys i s of h i gh r i sk 
profiles in order that intervention programs can be designed 
and implemented. 



Proj ec t Assump t i ons 



Project Object i ves 
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6. Provide on-line retrieval of information regarding class 
numbers, individual and group competency* and levels of 
performance. 

7. Prov i de con t i nuous objec t i ve i n format i on for educat i onal 
and career decision-making based on past performance, 
current status and future goals. 
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School systems have had rich Information environments for quite 
some time. One need only browse the list of information typically 
available in local school districts (see Table 1} to realize how much 
data is routinely gathered and maintained in some fashion. Until 
recently, however, many school districts lacked the financial and 
human resources (and often, the incentives) necessary to aake the 
available information serve as a viable component of decision-making 
in ongoing, enduring school improvement efforts. 

Obviously, times have changed as 1s evidenced in other papers in 
this symposium and related work (e.g.. Bank and Williams, 1983). The 
collection, analysis and maintainence of both achievement and 
non-achievement data within a comprehensive information system in 
order to (1) examine the functioning and impact of existing school 
programs, (2) monitor key school "health" indicators and (3) plan, 
guide and examine new instructional improvement initiatives is 
becoming a common feature of the local educational scene. These 
activities are both a sign of the times (once again, education and 
educational improvement matter) and an indirect by-product of the 
technology explosion and the improved sophistication of LEA research 
and evaluation personnel that resulted from evaluation and testing 
requirements associated with compensatory education programs. If 
Lyons, Doscher, HcGranahan & Williams (1978) were to replicate today 
their study of evaluation practices in school districts, lack of 
computer resources and expertise and staff technical skills would be 
much less severe problems. To a great extent the knowhow and 
wherewithal exist within school districts to make their available 
. information useful for decision-making with respect to a variety of 
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Now that the expertise and technology are possible, -ft seems 
appropriate to devote more attention to fine-tuning local school data 
analysis and reporting to make practices in these areas (a) more 
sensitive to the substantive decisions school personnel must make and 
(b) better reflect the methodological state of the art. To this end, 



this paper examines selected analytical issues that arise in making 
information obtained from the multiple levels (pupil /parent, 
teacher/class, school, district, community) of local school settings 
useful for decisions at the various levels. 

Even after a local school community has decided to undertake 
school improvement and allocates the resources to develop information 
systems for use in their efforts, questions remain about how the 
relevant information should be treated analytically. Although most 
districts routinely collect much of the pertinent data for school 
improvement efforts, these data are seldom analyzed and reported in a 
manner consistent with extant knowledge about the possibilities and 
limits of information from multilevel social structures. Since the 
same data (e.g., from standardized achievement or competency tests; 
parent, student, teacher, and administrator surveys; archival and 
demographic records) can take on different meanings when analyzed and 
reported at different levels (e.g., class vs. school ) and by 
different Indicators at the same level (e.g., the school average vs. 
the proportion exceeding a specified level of mastery), comprehensive 
information systems need to be designed to make valid, pertinent 
knowledge accessible for constituents at the multiple levels. This 
concern for knowledge utility and accessibility may require that the 
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same data be reconfigured (or simply reported differently) for 
different users. Moreover, multilevel analyses, albeit handled in a 
technically appropriate manner, must be reported in a form suitable 
for the analytically unsophisticated. 

In the remainder of the paper, we briefly discuss several 
methodological issues derived from a multilevel analytic perspective 
on local school improvement. This discussion draws heavily from both 
my previous work on analysis of multilevel data (Burstein, 1980, 1981 
in press; Burstein and Linn, 1982; Burstein, Linn & Capell, 1978; 
Burstein, Miller, & Linn, 1982) and from a longer conceptual synthesis 
that attempted to apply the general multilevel method framework to the 
local school improvement context (Burstein, 1983). The examples cited 
are taken from both applied research and from school district 
information and practices (both hypothetical and real). 

The Nature of a Multilevel Perspective 
Local school districts engaged in school improvement would appear 
to be settings particularly amenable to adopting a multilevel 
perspective regarding the collection, analysis, interpretation and 
reporting of information on school contexts, programs, and outcomes. 
To be internally consistent with this perspective, it is necessary to 
employ an analytical methodology that examines data from the different 
levels (student, classroom/teacher, school, district) of the 
■educational system and their interconnections. Such a methodology 
incorporates both a multilevel conception and an accompanying 
willingness to disentangle effects from a variety of sources and 
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levels (Figures 1 and 2 depict the data sources, data domains, and 
aggregation levels likely to be of interest in most school settings.) 
One must begin with a belief that no level of the educational system 
is uniquely responsible for the delivery of and response to schooling 
and thus substantive questions should rarely be confined to a single 
level (Burstein; 1980, in press; Rogosa, 1978). Thus a multilevel 
perspective and associated investigation focuses on the interface of 
Individuals and the "groups" to which they belong and on the 
Implications of this interface for understanding schooling. 

There have been, a number of syntheses of relevant research that 
focus on the theoretical, conceptual, and empirical bases for the 
Impact of the multilevel character of educational systems on the 
measurement and identification of the antecedents and correlates of 
educational performance (Hadaus, Airasian & Kellaghan", 1980; Barr & 
Dreeben, 1983; Bi dwell & Kasarda, 1980a, 1980b; Burstein, 1980a, 
1980b; 1983, in press; Cooley, Bond & Hao, 1981; Cronbach, 1976; 
Miller, 1981). These authors build a case that schooling can be 
better understood by, among other things, 

° utilizing an array of group-level (class, school, etc.) 
indicators that are potentially sensitive to differential 
performance associated with differential resource allocation 
strategies • • • 

° employing test analysis procedures that are likely to reflect 

instructionally sensitive variation in performance 
° linking a conceptualization of an educational process to its 

measurement and analysis at various levels 
° using analytical procedures that potentially identify effects 
at and within each level of the educational system 
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Each of these points Identifies analytical issues addressable 
from a multilevel perspective that becomes especially salient in 
information-rich school improvement contexts. In the remainder of the 
paper, we discuss and illustrate each point as it might arise as part 
of a school improvement effort. In doing so, we focus on group/ 
social /organizational rather than individual /clinical uses of 
information f or cl ass-1 evel , school-level and multi-school 
decision-making*-. 

Alternative Indicators of Group Performance 
Much of the analysis and reporting of achievement .and 
non-achievement (e.g., attendance, taking of advanced course-work) 
indicators of educational performance is conducted at the group 
level (typically school or classroom). All too often these 
group-level analyses employ only measures of central tendency (such as 
means and medians, average percentages). But when one's purpose is to 
understand schooling and depict its consequences, such measures of 
central tendency can hide important differences in the distribution of 
pupil performance and educational experiences- Under many 
circumstances, the distribution of performance from an instructional 
setting is likely to be as informative about the operation of 
educational processes as the group's typical performance (Brown & 
Saks, 1975; Burstein, 1980; Burstein & Linn, 1982; Cooley & Lohnes, 
1976; Klitgaard, 1975; Linn S Burstein, 1977; Lohnes, 1972; Spencer, 
1983; Wiley, 1970). 

The point here is that although a concern for achievement may 
drive instructional improvement efforts, it is important to keep in 
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the local school context and be more refined in its objectives. Under 
usual schooling conditions, a focus on raising the performance level 
of students around the middle of the overall performance distribution 
(e.g., say 40th - 60th percentile) will yield the highest gains in 
mean performance 2 . Yet such a thrust ignores just the segment of 
students who have the greatest needs. A focus on the performance of 
the lowest quarfcile, on the other hand, devotes instructional 
resources in a manner likely to. reduce the spread of performance (by 
establi. .*ng a performance floor or boosting more students over the 
minimal mastery point). Thus,, the multilevel principle that group 
means do not account for all relevant group-level information should 
lead to context-sensitive analyses intended to monitor instructional 
improvement 3 . 

Examining Spread 

The hypothetical data in Table 2 illustrates the value of 
multiple group-level indicators in simple comparisons of class-level 
performanceo While the students in both classes started at the same 
level on the pretest and had the same mean performance on the 
posttest, the variation in posttest performance is much larger for 
Class 2. Instruction in this class led to differential increments in 
learning gains {some students learned more than their counterparts at 
the same pretest performance in Class 1 while others learned less) 
while the posttest performance patterns in Class 1 reflect a more 
uniform distribution of learning gains. One. needs to know more about 
the specific circumstances to judge whether one class-level 
performance profile is to be preferred over the other, but the two 
profiles are definitely different and should be treated accordingly. 
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Comparing Distributions 

The data reported in Table 3 (taken from Spencer* 1983) 
illustrate the importance of trying to capture the entire distribution 
of performance in comparing groups (schools in this case}** If 
judgments about a school's effectiveness were based on a criterion 
score of 70, School C would be top ranked (because 20 percent of its 
students exceeded this score) while School A would be ranked at the 
top if the cutoff were either 45 or 40. This example might seem 
far-fetched, but it depicts what can happen when schools are judged by 
either the number of students taking advanced placement courses or 
their average SAT score versus judging schools according to the 
proportion of students passing a state's minimum conpetency or high 
school proficiency test* 

Measuring School Effectiveness by Subgroup Comparisons 

Another actual example derives from the recent school 
effectiveness literature* In his studies, Edmonds (1982) focussed on 
wi thin-school differences between lower SES and hfgher SES students in 
the proportion of students achieving mastery of designated educational 
objectives* Thus, if a substantial proportion of a designated group 
within a school performed at the prescribed level or if the 
differences in proportions achieving mastery were approximately equal, 
the school was judged "effective*. In essence, Edmonds 1 interest in 
the antecedents of effective achievement of low-income students caused 
him to depart from typical practice in school effectiveness studies of 
concentrating on school mean levels across all pupils. 



- 7 - 

249 



Ch aracterizing Survey Responses 

The last example is taken from the reporting of responses to 
survey items collected as part of A Study of Schooling (Good! ad, 
Sirotnik, & Overman, 1979) and later provided as part of a feedback 
package for a school (Table 4). Considering the student responses in 
Exhibit A first, the mean response to every item provides little 
indication of the diversity of feeling expressed by students. A 
significant number of students strongly dislike each activity, while 
in the case of "working with the whole class", an equal number 
strongly like this mode of learning. Students indicated definite 
preferences that, if reflecting true feelings, dictate against 
concluding that either it doesn't matter or students are undecided. 

A somewhat different point is illustrated by the teacher 
c responses to the organizational problem solving and principal 

leadership dimensions. The dimensions yielded equal means, but 
teacher opinions are certainly more divided with regard to their 
beliefs about principal leadership. The staff cohesiveness dimension 
exhibits both a higher mean and the virtual absence of "disagreement 11 
responses (only 3% scoring either 1 or 2). The means alone simply 
cannot capture what these data tell us about teacher perceptions. 

There are other studies and other group-level indicators (e.g., 
Burstein & Linn, 1981; Burstein, Linn, & Capell, 1978; Burstein, 
Miller, & Linn, 1981) that offer possibilities for improving 
explanations of the relationships among educational contexts, 
processes and outcomes. How to adapt at least the logic, and perhaps 
the methods, of alternative group-level indicators for use in local 
school improvement activities clearly warrants further consideration. 
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Instructional^ Sensitive Test Analysis 
Clearly, the emphasis in test construction, analysis, and 
interpretation is on individual differences in both classic and IRT 
psychometric treatments of test data. Yet it would seem that a 
concern for the sensitivity of test performance to instructional 
experiences would require test analyses approaches that reflect the 
organization of Instruction and the circumstances in which students 
receive instruction. There is ample evidence of the substantial 
variability across classrooms and schools in their content selection, 
emphasis, coverage, and method of coverage, not to mention the quality 
of instruction. Under these circumstances, multilevel examinations of 
the patterns of test performance (at the class and school level in 
addition to the student level) can be potentially valuable for 
detecting effects to background differences (e.g., prior learning, 
socieconomic and demographic differences), Instructional coverage and 
emphasis, and 'instructional organization (e.g., grouping and pacing 
effects). When these separate effects can be identified, it may then, 
be possible to construct measures and Indices which are sensitive to 
the context factors of instruction and describe performance 
accordingly. 

Me illustrate multilevel appraches to understanding and 
describing test performance in two ways. First we present a 
hypothetical example of the hazards of basing Interpretations of group 
(school, class) test performance on horizontal aggregates of many 
subskills and competencies. This example is followed by a brief 
report from an investigation of class-level patterns of responses to 
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test items that, in our view, illustrates what can be gained from a 
multilevel analysis of patterns of responses to test items. 

Aggregating Over Test Content 

The proper handling and interpretation of scores aggregated over 
individuals has received most of the attention in multilevel methods 
literature. Yet, in several respects, the logic holds as well for the 
content of instruction and of outcome measures. The choice between an 
emphasis on basic skills or on a broader array of knowledge- has much 
in common with the decision about which group or level is of 
interest. Just as a focus on low income students at the school level 
dictates interest in certain indicators of performance (and perhaps 
disinterest in others), judpents of the success of school improvement 
efforts can depend on the chosen level of aggregation over the content 
of instruction. It can also depend on the form of measurement of the 
content.- 

The concept of level of aggregation and measurement of 
instructional content can be depicted as follows. Following a literal 
interpretation of the dictums of Title I, School A institutes a strong 
back-to-basics effort, adding more drill and practice activities 
(spelling quizzes, timed math drills, memorization and recitation of 
poems). The additional time School A devotes to these activities is 
* obtained by foregoing most social studies, arts, science, and music 
instruction. School B, on the other hand, increases enrichment 
activities and attempts to enhance the breadth of its curriculum- 
through dramatic play in its social studies work (e.g., 



various classes enact political campaigns prior to major elections, 
operate mock city governments, "live" through the experience of the 
Pilgrims, etc.). The teachers in School B tie in most lessons in 
reading, mathematics, and writing with these dramatic assignments but 
leave little time during school hours for drills on math, spelling and 
language arts facts. 

Schools A and B make clear their instructional preferences. 
However, unless the array of non-teacher made tests (annual 
standardized tests, state assessment, district continuum) given in the 
schools are unusual, aggregate scores from these tests will 
differentially reflect the instructional emphasis and quality of the 
two schools. If two-thirds of a test's reading questions were devoted 
to word identification, vocabulary and spelling and its math questions 
to basic facts and computations, total test scores might make it 
appear that School A is more effective because its students uniformly 
mastered their narrower curriculum material that predominated in total 
scores, while School B's students performed more inconsistently on 
their facts and mechanics. (Presumably, School B would do much better 
on more complex comprehension, reasoning and interpretation material 
that is covered by a more limited portion of the test.) 

Unfortunately, this fictional scenario probably occurs all too 
often in current efforts to determine the content of instruction and 
its measurement in school improvement efforts. The premise here is 
that the level of refinement in distinctions about instructional 
content should be an explicit choice and the measurement of the 
consequences of instruction should be sufficiently refined to reflect 
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both desired and unintended content distinctions* Aggregation over 
content in the scores from multidimensional tests is potentially 
hazardous if the intent is to determine what has been taught and how 
well. 

Examining Response Patterns at the Group Level 

Examinations of the patterns of students' responses to test Items 
across occasions and across groups (classes, schools, etc*) can be a 
particularly informative means of deriving explanations of test 
performance. A reanalysis of selected test item data from the 
Beginning Teacher Evaluation Study (BETS; Fisher et al., 1978) 
suggests how response patterns might provide information about 
instructional differences. Miller (1981, 198*) examined answers 
chosen by 123 students from 21 fifth-grade classrooms to the 15 items 
from the fractions subtest on two occasions (prior to (Occasion B) and 
following (Occasion C) most instruction in this subject area). He 
classified the test items into four subtopics: adding of fractions, 
subtracting of fractions, equating fractions, and solving fractions 
with algebraic unknowns (e.g., X/3 - 6/9. What is X?). Tables 5 
through 7 present a subset of Miller's results. 

First, the intercorrelations of item performances between classes 
and among students within the same class (Table 5) clearly indicate 
the effects of differential topic coverage across classes and within 
classes. Virtually all of the high item intercorrelations occur in 
the between-class analysis. This reflects differential coverage of 
topics across classes* Those classes which taught addition of 
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fractions also taught subtraction and those classes with high (low) 
performance on addition also tended to exhibit high (low) performance 
on subtraction,. The classes which taught algebraic unknowns 
apparently didn't teach (or unsuccessfully taught) addition and 
subtraction of fractions. On the other hand, there was virtually no 
intercorrelation among item performance within classes. There was 
very little tendency for a student to perform well across all items. 
Different students answered the various questions correctly 5 . 

Table 6 present results from a selected set of classes with 
distinctive response patterns. Since time allocated to fractions 
instruction per week is also reported, it is possible to also 
speculate as to whether classes received any fractions instruction at 
all. Note especially the contrasts in performance. For example, 
classes 8 and 5 had the same posttest score and approximately the same 
gain, but class 18 did so by covering and mastering every topic except 
addition of fractions (these students exhibited the same performance 
pattern at pretest so it is likely that the teacher simply did not 
check to determine whether students had mastered the mechanics of 
fraction addition after the pretest.) Class 5 likely depicts the more 
typical pattern of mastery of simple addition and subtraction of 
fractions and virtually no other content coverage. 

Classes 3 and 16 represent another interesting contrast. These 
two lowest scoring classes got there in different ways. The teacher 
in CI ass 16 devoted a substantial amount of time to the coverage of 
fractions but virtually all the time seems to have been spent learning 
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about algebraic unknowns. Class 3 devoted almost no time to the 
fractions topic (the majority of the class did not attempt to answer 
the questions at either pretest or posttest.) But there was some 
differentiated content coverage as a few students in Class 3 mastered 
all four subtopics. 

Table 7 presents information derived from examining the 
class-level variation in the actual response alternative selected. 
This analysis clearly suggested systematic differences across classes 
in the types of errors students made. The. two addition-of -fraction 
items demonstrate that students in several classes apparently never 
learned not to simply add numerators and denominators. Several 
classes did not appear to teach the expansion of fractions or taught 
fraction reduction in such a way that the students did not grasp its 
flip-side. The algebraic unknowns items exhibit the greatest variety 
of class-specific confusion. Obviously, some aspect of instruction is 
responsible for the systematic misunderstanding of how to do this type 
of problem. It may simply mean that in the absence of instruction on 
algebraic unknowns, students from a given class facing a novel task 
(for them) try to apply some other algorithm they have learned. 

This example of investigating patterns of test performance barely 
scratches the surface of modern psychometric work on item response 
patterns (See, e.g., Harnisch, 1983; Harnisch & Linn, 1981; Sato, 
1975, 1980; Sato & Kuto, 1979; Tatsuoka fi Linn, 1983), much less 
recent advances in information processing models for test design and 
interpretation (e.g., Baker & Herman, 1983; Brown & Burton, 1978; 
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Birenbaum * Tatsuoka, 1982; Curtis 4 Glaser, 1983; Davis, 1979; 
Tatsuoka, 1983). Yet it is clear that the combination of this Modern 
orientation toward test design, better psychometric indices of 
response patterns, and analyses that take into consideration the 
levels at which instruction is delivered can provide better 
information about test performance for decision-makers. 

Multilevel Measurement of Edu cational Processes and Context. 

The principle that the same observable variable can measure 
different constructs at different levels of analysis is well- 
established (Burstein, 1980a, 1980b; Burstein. Fischer, and Miller, 
1980; Capell, 1981; Cronbach, 1976; Sirotnik, 1979). A few examples 
serve to emphasize its ubiquity in educational "research. Take, for 
instance, the standard measures of socioeconomic background typically 
found in studying schools. At the individual level, they may properly 
convey the parental investment in the individual child's learning. 
Once aggregated to the school level, social background measures also 
reflect the community context (e.g., wealth, urbanism, commitment to 
quality education) which in many cases conditions the resource 
allocations to schools. Within an educational level, relative social 
background positions students within a potential status hierarchy 
(e.g., a big fish in a small pond) That can affect their experiences 
(Burstein, 1980a, 1980b; Burstein et al., 1980).' All three measures 
of social background may be important in understanding the experiences 
and performances of students but they do represent distinctly 
different mechanisms. 
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In a reanalysis of data from an observational study of the 
factors influencing student learning, Burstein (19cO) demonstrated how 
the interpretation of a measure of the relative amounts of student 
learning tasks judged easy changed as -the analysis shifted from the 
student to the class level. Students" success rates in learning tasks 
at the individual level captured proximal student ability and thus 
werepositively related to student performance. At the class level, 
this same observational variable reflected teachers* policies with 
regard to task difficulty and in many instances exhibited negative 
relationships with student outcomes. 

The problems of change in variable meaning across levels are 
particularly evident in the literature on organizational and 
educational climate (e.g., Capell, 1979; Sirotnik, 1979). The 
distinction between a specific student's perception of classroom 
climate, which reflects both absolute and comparative aspects of 
individual personality and perception, and the average perception of 
the class, a normative measure of the instructional environment, is an 
important one. Whether the "organizational" or the "psychological" 
aspect of the climate is most salient in a given context is unclear. 
Capell (1981), for instance, construed aggregate responses of teachers 
within schools on scales purported to measure .the degree of innovation 
and teacher influence as indicators of the atmosphere and 
organizational structure of the school program (See Table 8). In 
contrast, the individual teacher responses, relative to the responses 
of other teachers in the school, were interpreted as indicators of the 
teachers' sense of personal efficacy. That the effects of aggregated 
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and individual measures on pupil outcomes were opposite in sign and 
consonant with expectations reinforces the need for a better 
understanding of how aggregation affects the measurement of program 
and process characteristics. 

The studies cited above are important for our present purposes in 
two respects. First, they demonstrate the value of multilevel methods 
in educational research and evaluation. Second, and more importantly, . 
the measures used in these studies — socioeconomic background 
indicators, survey responses from students, teachers and parents, 
classroom observation data — represent typical information about 
educational processes and contexts that are or can be gathered in 
local educational settings. Apparently, these measures can serve as 
indicators of a variety of constructs — home resources, community 
resources, organizational structure and atmosphere, personal efficacy, 
classroom and school climate, appropriateness of content, student 
level of functioning — that are important in understanding and 
improving schooling when the linkage between the level of aggregation 
of measures and the construct of interest is clear. 

The relevance of these concerns about the shift in variable 
meaning across levels is particularly pertinent to a variety of 
investigations of the effects of schooling that a number of local 
school districts have undertaken. Several school districts have 
conducted their own studies of school effects (e.g., Kean, Summers, 
Raivetz i Farber, 1979 (Philadelphia); Ramey, Hillman « Mathews, 1982 
(Seattle)) and school effectiveness (e.g., White & Kemp, 1976 
(Atlanta); Gastright, 1977 (Cincinnati)). The methodology they employ 
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mirrors the practices of large-scale investigations of school effects 
and school effectiveness. The LEA-based studies seem no more- nor 
less resistant to problems in measuring the variables of interest at 
the appropriate levels, and incorporating them properly in their 
analyses, than their large-scale, multi-site counterparts (see 
Burstein, 198CU Madaus, Airasian & Kellaghan, 1980; and Purkey & 
Smith, 1982 for discussions of methodological problems with these 
types of studies). If local educational agencies use these 
investigations to guide their decisions about instructional 
improvement programs and other school renewal activities, then one 
would hope that inattention to specific concerns about appropriate 
level of measurement and its relevance to construct-indicator match 
would have limited impact. 

Analytical Methods for Disentangling Multilevel Effects 
Two recent classroom studies of beginning reading demonstrate the 
value of decomposing the variation of students" instructional 
experiences and performance into variation associated with subgroups 
within classrooms. Barr & Dreeben (1983) focussed on content coverage 
as a variable and found that it varied mainly between reading groups 
and not between teachers. Hot surprisingly, students performance also 
varied primarily between reading groups. 

In the Texas First Grade Reading Group Study (Anderson, Evertson 
S Brophy, 1979), the focus was on teaching behavior variables such as 
teachers* selection of students to read (e.g., non-volunteer 
selections), types of student responses to teacher questions, and 
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types of teacher feedback. In a secondary analysis of the study data, 
Martin, Anderson, and Veldman (1980) decomposed the variation in 
student achievement into effects of teacher behaviors at three levels: 
students within reading groups, reading groups within classes, and 
classes. Most of the significant relationships were for 
students-within-reading groups. Also teachers tended to change their 
selection strategies across reading groups within their class. An 
analysis of class means only would have missed the effects of 
teachers' differential activities across reading groups and the 
differential impact of teacher behaviors on the members of specific 
reading groups ♦ 

There is no need to provide additional rationale for and describe 
developments in analytical methods for disentangling multilevel 
effects. As with the 'studies of reading, the substantive 
investigations described in earlier sections typically combined a 
better conceptualization of the multilevel character of educational 
data with analytical machinery adapted to the substantive questions of 
interest rather than molding the theory to meet the conditions of the 
statistical procedures. Generally, the analytical procedures employed 
were the familiar ones, but these tools were used in a variety of ways 
that better mirrored the process of schooling. Typically, better 
analysis of multilevel educational involve disentangling influences at 
and within each level of the educational system by conducting multiple 
analyses or a common analysis with measures collected from (or 
aggregated to) multiple levels* 
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Currently, methodological research on methods for analyzing 
multilevel data is focussing on relatively sophisticated procedures 
that require stronger assumptions b* t at the same, time, are more 
robust to typical shortcomings in social science data (e.g., missing 
data, asymmetrical distributions, measurement error, 
heteroscedasticity, resistance to outliers. Relevant work is reported 
in Aitken, Anderson, and Hinde, 1982; Burstein & Gustafsson, in 
progress; Goldstein, In progress; Mason, Wong & Entwistle* 1983; 
Rachman fi Wolfe, 1983; Schneider & Treiber, 1984. While little of 
this work will have direct bearing on routine analyses to guide local 
school improvement, they may provide a better means for 
re-interpreting results from studies of the effects of schooling in 
ways that are more consonant with the perceptions of local school 
personnel. If so, such reanalysis would provide better support for 
effective local practices and less ammunition for school critics. 

Concluding Comments 
The value of a multilevel perspective for understanding the 
effects of schooling is becoming a more commonly held perception 
across a wide array of educational professionals (researchers, policy 
makers, and practitioners). We have attempted to illustrate several 
ways in which such a perspective might lead to more sensitive and 
sensible data analysis that are better suited to school improvement 
efforts. 

To some degree, our efforts bog down when we attempt to shift 
from studying and understanding the concept of multilevel analysis and 
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turn to its practice. School cultures involve a variety of sources of 
information and a number of constituencies (e.g., teachers, 
counselors, administrators) with clearly demarcated responsibilities 
that m(ght be able to use properly collected, appropriately analyzed 
and routinely accessible information. But little is known about how 
school building personnel operate in a context with high-quality, 
timely, pertinent information, either because these conditions do not 
exist or no one has yet documented how school -level personnel respond 
under such ideal information conditions. If the other symposium 
papers are indicative of what is now possible with respect to the use 
of comprehensive information systems in schools, we won't have to wait 
very long to determine whether the promise of a multilevel perspective 
toward the analysis and reporting of school data is real or illusory. 
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FOOTNOTES 

iThis distinction is a useful one. Individual /clinical uses of 
information include such activities using test data for individual 
diagnosis of learning problems, placement decisions, individual 
student counseling and guidance activities, administrator supervision 
of individual teachers and similar individual personnel matters 
(hiring, course asignments, promotions, etc.). Group/social/ 
organizational uses refers to the myriad of ways in which data from 
individuals are aggregated and organized to characterize/depict/ 
understand the functioning and behavior of groups of individuals. 
Class-level test performance and background profiles, subgroup (e.g., 
by ethnicity, sex, grade-level, curriculum track, SES) information; 
course enrollment and course-taking patterns across subject matters 
and subgroups of students are all examples of the latter. Most of the 
analytical developments pertinent to this paper deal with the latter 
type of use. 

2 There are both substantive and technical reasons for this. 
Substantively, students at higher percentile levels have a low 
ceiling; that is, much of the new material they might learn is not 
likely to be reflected in substantial improvements in the test 
performance because this material typically is not well -represented on 
the test. Low performance, on the other hand, may require substantial 
resources to boost performance above, say, the 50th percentile and 
thus draw off resources from students in other parts of the 
distribution. For example, concentration on the skills needed by low 
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performers may lead to more wait time and inefficiency for high 
performers (unless the latter are allowed to "work ahead on their 
own")." Performers in the middle of the distribution are likely to 
benefit from the focus because they have sufficient room to grow and 
may simply require a bit more targeted instruction to clear up certain 
misconceptions (see later example about fraction addition) or to 
acquaint them with topics not previously covered. 

The technical side of the argument is the well-known relationship 
between raw score points and percentiles. It takes more raw score 
change to move up a given number of percentile points in the tails 
than in the middle of a normal distribution. Thus gains in knowledge 
in the middle of the distribution boost the average percentile more 
rapidly than gains in either the upper or lower tails. 

3 0ne does not have to be devoted to compensating for the inadequate 
performance of low income pupils to derive benefits from interest in 
the distribution of performance rather than simply its level. In 
their annual reports, the California Asessment Program provides 
schools with the quartile distributions of the performance of 3rd 
grade students along with a variety of mean indicators (overall and 
for various demographic subgroups). A former principal of a suburban, 
typically higher performing school pointed out that while the school's 
overall performance each year (typically above 70 per cent correct, 
which is above the 90th percentile statewide) didn't tell him 
anything, he did keep track of the number of children that fell in the 
lower quartile each year because this meant that there were still 
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students who needed to improve. Thus, even high achieving schools can 
benefit from an awareness of the functioning of their weakest students 
and school mean performance doesn't typically capture this type of 
information. 

^Spencer's paper (1983) considers a number of statistical problems 
associated with the typical use of test scores for comparisons of 
outcome differences among groups. He highlights the problems 
associated with the ordinal properties of most metrics used to measure 
outcomes and presents a case for switching to indices of "stochastic 
ordering" when attempting such comparisons. This shift would 
certainly be in the direction of maintaining more distributional 
information in group-level analyses. 

5 The exceptions to the lack of correlation across items within classes 
are typically for those items that are essentially parallel (e.g., 
items 1 and 2; item 6 and 7; and items 11, and 13) or involve 
analogous straightforward topics such as addition and substraction 
items containing a common denominator. 
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Table 1 

The Types of Information Routinely Collected 
(or collectable) in School Districts 

A. Demographic/Archival 

1. Student demographics-age, sex, ethnicity, home language, 
parental occupations and employers, eligibility for AFDC, 
reducted price lunches, medical histories, home address,' 
mobility (how long in particular residence) parental 
education, family size 

2. Teacher and building-level administrator backgrounds - age, 
education, previous employment and educational history, 
special certification and subject-matter expertise 

3. School building characteristics - information about physical 
plant (e.g., age, capacity, particular resources), 

4. Student body and community composition-ethnic composition, 
neighborhood wealth, community involvement in neighborhood 
schools (e.g., PTA membership) 

B. Financial 

5. Payroll expenditures 

6. Materials and supplies 

7. Equipment 

8. Maintenance 

9. Special programs (e.g. entitlement programs, staff 
development, remedial services, counseling and guidance) 

10. Transportation 

11. Safety and Security 

C. Testing 

12. Standardized norm-referenced tests 

13. Criterion referenced testing 

14. Minimum competency and proficiency testing 

15. Group and individual ability and aptitude testing - done 
typically to determine pupil eligibility for special programs 
and placement decisions 

16. Teacher-made tests and curriculum embedded tests 
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D« Program Characteristics and Participation 

17. Special program participation — availability and staffing of 
special programs at local school sites 

18. Curriculum information — curricular packages and texts used 
in classrooms, topic coverage from continuum {assumed and 
measured) 

19. Course taking patterns — information from student cummulative 
records and from prescribed offerings 

20. Grading practices — teacher reports of student grades 

E. Student Performance, Participation, and Behavior 

21. Grades by content area 

22. Participation in extracurricular activities by types 

23. Awards — e.g., scholarships 

24. Absenteeism and tardiness 

25. Reported disruptive and inappropriate behavior 

F. Affective, Attudinal, and Observation Information 

26. Student responses to surveys about class and school 
environments and other aspects of their educational experience 

27. Teacher measures of classroom and school climate and 
activities 

28. School building administrator measures of school climate and 
activities 

29. Parental surveys of perceptions and support of school 
activities 

30. Parental participation in school activities (e.g., volunteers, 
fundraising attendance at school functions, scheduled 
conferences) 

31. Administrator observations and evaluations of teachers 

32. Teacher observations of other teachers 

33. District personnel's observation and interviews of building 
personnel 

34. Surveys of graduates to determine occupational and educational 
status 

35. Information about student dropouts 

* 
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G. District Evaluation Reports 

36. Routine annual reports to board and federal and state agencies 

37. Evaluation of specific educational changes 

38. Instances of local school assistance by type and disposition 



6 Source Burstein, L. The Use of Existing Data Bases in Program 
Evaluation and School Improvement (1983) 



9 

ERIC 



- 27 - 



Table 2 



Hypothetical Test Results For Two Classes 
with Equal Pre-and Posttest Means, 
Equal Prestest Variances, but Unequal 
Posttest Variances 



t*c+ c,™ ?r etes J Frec l u encies Posttest Frequencies 

Test score Class 1 Class 2 Class 1 Class 2 



7 0 0 



0 3 



6 0 0 2 3 

5 2-2 6 4 

4 .6 6 9 5 

3 9 9.6 4 

2 6 6 2 



1 2 2 0 

0 0 0 0 



3 

• 3 
0 

Heans 5f«i s 3 X. 2 -3 7^ = 4 7. 2 = 4 

Standard 1.08 1.08 . 1.08 2 17 

Deviations 



Source: Burstein, I. & Linn R.L. Analysis of Educational Effects from a 
Multilevel Perspective:Disentangling Between- and Within-Class 
Relationships in Mathematics Performance, CSE Report No. 172, 
University of California Los Angeles, Center for the Study of 
Evaluation, 1982 
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Table 3 . Percent- of Students Scoring at Least X 



School 


40 


45 


Score x 
50 


55 


70 


A 


90 


80 


60 


50 


10 


a 

• • 


85 


75 


55 


48 


14 


c 


83 


70 


65 


48 


20 


D 


71 


60 


40 


20 


3 


• 

* 


• 

* 


• 


• 






• 




• 
• 


• 
• 


• 
* 


• 
• 
• 



Source: Spencer B. D. On interpreting test scores as social Indicators: 

Statistical considerations, Journal of Education al 
Measurement . 1983, 20(4), 317-334. 
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Table 4 

Examples of Responses to Survey Items where Mean response is insufficient 
to reflect distribution of responses. • 



Exhibit A Responses. of 26 students from a siogle secondary school class . 

Percentage of students responding.-.. - • 
Like Like Dislike Dislike 

very much somewhat- somewhat ■ very much Mean 



Working with the . 





. 23 


38 


15 


23 


2.4" 


Tell in my own- words 














: 04" 


23 


31 


42 


3.1' 






54 


19 


23 


2.fv' 


Do research and write 














04 


23 


31 


42 


3.1 



Exhibit B— * Responses of 34 teachers from a single school to items measuring 
teachers perceptions of the work environment. The 77 items were 
combined into 3 scales (dimensions of the work environment) which 
were labeled "organizational problem solving", "principal leadership" 
and "staff coh>esiveness". Items were answered on a six-point agreement 
scale and the school mean and distribution of teacher scores (average 
response to items from a given dimension) are reported below: 



Dimension 

« • . "v. • ; * 


" Mean 


ii 


• Teacher Distribution (Z) 
_3 _4 _5- \_6 


1. Oranizational Problem Solving 


" 3.4 


3 


12 


41 


29 


12 ; 3 


2. Principal leadership 


3.4 


12 


18 


24 


26 


9 12 


3. Staff Cohesiveness 


3.7 


0 


3 


44 


35 


15 3 



a . . ... . 

Data and questions taken from an example feedback package from A SXudy of 
Schooling (Goodlad, Sirotnik et. al.) which also appeared as Appendix B in* * 
Sirotnlk & Burstein (1983). 
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Item intercorrelationa between classes (lower triangle) 
end within classes (upper triangle) on occasion C t 



1. 
i 

3 
4" 

S 



6 
7 

e 

9! 
10 



Subtraction 
2 3£ 



.86 
.19 
.62 
.43 



-.75 

.28 
,53 
.41 ■ 



.07 
.00 

.50 
.36 



.33 • -,03 
.29 -.08 



.30 
.39 



.06 
.03 



.68 .42 .34 .43 .27 

-.52 \29 ".55 .48 "'.38 

.49 • ■ .29 " ".30 .48 « .42 

.53 137 ' '.33 .45 .69 

.56 .28 .05 .40 .49 



.55 .45 .33. .56 .64 
'.41 . .37 '.60 .74 ,5Z 
.50 ' .51 .19 .45 .60 



Addition 



10 



.51 .29 

. .64 ' .33 

.17 .29 

.25 .30 



.09 * .01 . ' .23 

.07 -.00 .16 

.26 .25 t .23" 

.23 '.22 ' .19 



..15' -.00 ,37 ".40 ."-.01" 



.86 
.66 
.64 
.61 



.59 

.64 

.59 
.62. 



.14 .15 
.29 4 ;25 
.67 

.75 

.53 .69 



.36 
.50 
.24 
.19 



.42 
.26 
.26 



.45 
f 38 
.16 



.47 
.57 

.59 



.58 
.41 
.51 



.51 
.23 
.29 



• 1 
i 

Equating 
11 12 .13 



.23 
.19 
.29 
.23- 
.19 



.21 
.12 
.03 
.05 
.18 



.19 
M9 
.35 
.21 
.12 



.69 
.56 



;50 
.69 



Algebraic 
Manipulation 



.22 
.02 
.05 
.15 
.19 



14. 

.26 
.29 
..16 
..40 
.16 



,08 . .07 
.16 ' . 08 
.29 • ^23 
..25 .16 
.06* •• ,13 



.3? 
,38 



.05 
.16 
..35 
.23 

.27" 
.29 
-.17 



15 

.20 
.18 
.06 
.26 

-.01 

7l9~ 
.18 
.20 
.24 
.23 
,23 
.35 
.29 



.18 .07 -.07 .36 .31 
.29 .18 -.05 .33 .39 



-.10. --.12- .41 .35 ..12. 
110 ' ' ,08 " .48 .53 .33. 



.47 .50 ,55 
.64- '.48 .49 



.81 



.37 



Source: 



M.D. Miller, Item Response and Instructional Coverage, »1984. 
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• - Table 6 

Clxssrooas exhibiting distinctly* dui-livil pattern* of performance on fraction iteas.* 

Clase ^ Allocated xig« Fog t test Cain 
» t . * **nk Mean Kank* Mean Rank 6 g*tura of the jtew response pattern - * 



0.8 -119 1 7.5 10*5 5-0 7 Mistered equating, algebraic manipulation, end sicple 

subtraction. Only problem at posttest was addition 
of fractions (adding numerator and denominator) where 
only 2 correct out of 30 response* (6 students responding 
to -5 lte&s)* 



5 


77 


5 


7.5 


1Q.5 
« 


5.2 
* • 


5 


Students can da simple addition and subtraction items 
but not more complicated ones. Ko work on equating, 
float covarage of algebraic manipulation but >esically 
don't understand the concept. * 


SO 


•114 


2 


12.5 


1 


6.5 


1 


Ko problems at post test except on algebraic manipulation. 


12 


87 


4 


11.3 


3 


6.3 


2 


- i ■ i * 

Host mastered every thing but 1 or 2 didn't master 
any fraction topic (possible differential coverage). 


. 11 


" .' 77 


' 6 


12.3 


2 


■1.7" 


15 


lots of time spent on fractions vith not much new 
learning, at least on the topica measured by the test* 


8 ' 




14 


9-0 


7 


5.6 


3 


Success on topics covered (sub traction, equating, atopic 
addition). Addition other than aicple cosaon denominator 
xaisl^arned or not taught. 


26 


46 


10 


6.4 


13 


5.0 


7 


Fraction curriculum at low level. Host learned only r 
. addition and cub traction with'coomoa denominator. One 
or two students learned more/ * . 


"14 


29 


12 

« 


5.3 

• 


15 


3.7 


9 


-*■ * — ■ i — ■ — ■ — ■ — - 

Differentiated teaching and learning. Soma mastered 
everything, some mastered addition and subtraction 
vith comeon denominator, others mastered' (probably 
covered) nothing. 


16 


52 


7 


2.8 


. 21 


1.5* 


16.5 


• * - 

.Students in .this class only mastered algebraic manipulation 
They did not answer any other questions on either pretest 
or posttest (with one or two exceptions). 


3 


3 


19 

* 


4.8 


20 


3.3' 


10.5 


Differentiated content coverage , almost all on algebraic 
manipulations, some students mastered post topica. 



-Based on scores of 123 fifth graders from 21 classrooms in the Beginning teacher Evaluation STudy.These 
analyses are reported in Killer (1984). 

minutes per week spent oh fractions during period between pretest and posttest. " 

« • 

Sased on ranking of ncan tls . aU o c .ted to fraction, acro 3 a the 21 classroom, " 

d Ba.ed on posttest average.^ a saaple of ^proximately 6 students fro, ea'ch class ' ^ 

a, ...... ^ class. Hsxisua possible score vas 15 

^>a.ed on ranking or «.„ achieves score at posttest for the 21 c i 0 „es. ' ? 

class eean difference between total scores at the pretest and posttest." 

E Bnscd on ranking of the neaa gains for all 21 classrooas. 
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Tible 7 

Test Items exhibiting distinctive class-level patterns of pfcudent performance* 



Question 

-4 



•Alternatives 



i£ 2 i-i 

..8 X 8 Z ^12 



Pretest p-value-.ll 
Posttest p-value».35 



2 a. 1 

3 3 



3 
6 



3 ~ 



Pre p-value «.U 
Post p-value-.26 



3 ± 5- 



28 J 1 4 
35 14 ' 7 



Prep-value » ,D7 
Post p-value~* 24 



^ Nature of the Response Pattern 
As students learned about subtracting frac- 
tions, many did not learn how to obtain a 
common denominator before subtracting. This 
resulted in some classes systematically - ; 
choosing alternative 2> probably because 
they had been taught how to handle fractions, 
with a common denominator ((occurred in 5 
classes at posttest). 3 classes exhibited 
mdsteyy at posttest. 



The mbst common error for both questions is 
adding both the numerator and denominator. 
This problem was not resolved at the posttest 
for several classes (9 classes on the first 
Item, 13 on the second). Both questions require 
change to mixed fractions -which students in 
many classes apparently were not taught. 

?our -lasses exhibited mastery of the first 
itam and 2 classes exhibited mastery of the 
second item at posttest 



1 
2 



2 
3 



2 
3 



JB 
12 



3 
6 



Pra p-value »■ ,32 
Post p-value~ .55 



3 
4 



Pre p-value ■» .15 
Post p~value» .32 



2 
5 



3 
9 



4 
9 



6 



Students in some classes could not expand 
fractions at the posttest (5 classes on each 
item); They simply added 1 to both numerator 
and denominator. 

8 classes mastered the first" item at posttest 
and 2 classes exhibited mastery of the second 
item. 
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What does N equal? 

2 
7 



~ 2l 3 16 



Pre p-value « .20 
Post p-*value- .33 



Lpts of class-specific confusion. Only 1 class 
exhibited mastery. Pour classes systematically 
chose alternative 1, 2 classes systematically 
chose alternative 2 and 1 class systematically 
chose alternative 3 at the posttest. 

9 classes mastered the somewhat easier item 
3/8 « 6/N (posttest p-value -.53) with only 
limited systematic ".errors* 



These are test results from selected fraction items given to 123 students in 21 
fifth grade classrooms before and after instruction on this topic(in most classes) 
The data were collected as part of the Beginning Teacher Evaluation Stud/ (Fisher 
Sio?* ? )# TU " re3ult8 ^orteA here arc based on a dissertation by Miller 
IX98.L j are reported more thoroughly in Miller (1984) . 
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i TbM a Q * 

• ^^VL^T^S, 5 ™^ 1 REAB1S0 ACHIEVEMENT ON 
TEACHER AND MINISCHOOL LEVEL SURVEY VARIABLES 3 > b 



Multilevel Regression 



Variable Name 



Teacher 
Level 



Minischool 
Level 



Class Level 
Regression 



Staff Cohesion 

Common Minischool 
Policies 

Teacher Autonomy 
Teacher Influence 
Principal Influence 



c 



-.225 
( .487) 

-.336 
(1.445) 

1.652 
(1.601) 

.660 
(1.714) 

.545 
(1.518) 



.679 
( .849) 

.360 
(1.155) 

-2.256 
(1.323) 

-1.330 
(2.292) 

-.961 
(1.843) 





.197' 


( .536) 




-.057 


( 


.371) 




.563 


C 


.696) 




-.135 


( 


.508) 




.004 


( 


.000) 
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^standardised regression coefficients, t statistics in parentheses. 
Sas^ua^ eqUati0n ^ 6StWed USin * the ««»d of weighted 

Wh6re . ? - t ^n i )- 1 kn 1 » .N^kn. ; - 

? is the vector of regression coefficients; X is the matrix of independent 
variables; k is the number of classes; N is the total number of students-" 
and n, is a diagonal matrix of class sizes. Use of the matrix « insures that 
each classroom in the analysis will be weighted by the number of students 
contained in it, while the overall degrees of freedom for classes will be 
preserved, 
c 

This variable was measured such that a ncsativA ™ rt *<r,- - 
greater influence. negative coefficient represents 

Source: Capell, F.J. A_study_of_alt^rnatives in An,ow 

Santa Monica, oJ^Tc^^^^ 0/6-HIE, 
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Personal 



0 

o 
u 

3 
o 

ft 



o 

a 



• Demography 

• Reasons for entering edu 
cation profession 

» Teaching experience 

• Educational beliefs 



Demography 
Self-concept 
Educational aspirations 



Demography 

Years lived f n community 
Political beliefs 



«3 i 



I ! 



Data Domains (Examples Onfy) 
Cfass School 



•Relative amounts of time 
spent on Instruction, be* 
nayior control, and routines 
tiv« t)ehav?0fal o^Jec 

• Frequency of certain 
learning activities 

**lV W * amounts of time 
spent on Instruction, be- 

•Difficulty of class content 
•Frequency of certain 
learning activites 
-Class "climate-. 



-Relative amounts of time 
spent on instruction, be- 
bayior control, and routines 
•Use of corrective "feed- 
back 

• Use of open versus closed 
questions 

•Instructional time spent 
with total class versus indi- 
vidual versus groups ' 



•Relative importance of 

22? funct,on * (social, in- 

ItiSf 10 ? 1 ' pers °n»'. and vo- 
cational) 

•School "climate- or work 

environment 

~ Major problems 

tJ3X* my ot wucation 
(ability, race, sex) 



Schooling 



of 



Relaiive Importance 
school functions 
" Evaluative rating 
Major problems 
• Equality of education 

»CfvlSs aCy ° f coun5e,in 0 
Subject-area preference* 



of 



Relative importance 
school functions 
" Evaluative rating 
Major problems 
Equality cf education 
Involvement in activities 
and decision making 

•Objectionable learning 
materials * 



of public j 



• Desegregation 
•Fiscal support 
education 
•Teacher* unions 

• Minimum competency 

• Role of global education 
in the schools • 



Desegregation 
• Rolt of Job experience In 
schools 

•Value of school* 



Desegregation 

Fiscal support of public 
education 
•Teachers unions 
• Teachers* salaries 
~ Minimum competency 

Rol* of global education 
in schools 



**re u*ed , n coNcct.ng data frc*> teaciws, stents. 



Figure i 
The Schooling Terrain: Hap One 
SOURCE: Goodlad, Sirotnik & Overman, 1979 



277 

- 35 - 



DATA DOMAINS 



Individual 



Class 



School 



District 



Data 



Personal 
(Individual) 



Categories : CA M 



Data 

Sources : 

Students 
Teachers 
Administrators 
Parents 

Students 

Teachers 

Administrators 

Parents 

Classroom 

Students 

Teachers 

Administrators 

Parents 

Classrooms 

School 

Students 

Teachers 

Administrators 

Parents 

Classrooms 

Schools 

District 



Instructional 
(Classroom) 

~C A ~ 



Institutional 
(School) 

CAM 



Societal 
(Schooling) 

C A H 



Data Categories ? 
C « Circumstances 
A « Activities 
K • Meanings 



The Schooling Terrain? Hap Two 



Source :Sirotnik K. & Burstein L. Systemic Evaluation, Los Angeles i University of California 

Los Angeles, Center for the Study of Evaluation, October 1983, 
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USING VS. BEING USED BY 
SCHOOL INFORMATION SYSTEMS 

Kenneth A* Sirotnik 
University of California, Los Angeles 

For the past dozen years, I have spent most of my professional work 
life thinking about, writing about, and using comprehensive data bases on 
schools and schooling. About two thirds of this time was spent working 
with John Goodlad and my other colleagues on A Study of Schooling— a study 
that attempted to develop a comprehensive, data-based description of what 
goes on in schools and how people feel about it— all in an effort to draw 
some working hypotheses about how things might happen differently (Goodlad, 
1983). More recently, a few colleagues and I at the Center for the Study 
of Evaluation and at the Laboratory in School and Conmunity Education have 
paused to reflect upon the role of comprehensive data-bases, less from the 
standpoint of doing a research study across schools, and more with an eye 
towards how information can be used for local school improvement and 
change. 1 

I note this brief, personal history only to support my commitment to 
the whole idea of comprehensive, school-oriented information systems. Yet 
as excited as I am about the potential use of the idea, I am worried about 
the potential of its abuse . It seems increasingly clear to me how people 
can use school information systems. And it seems increasingly clear how 
such systems can use people. If the latter turns out to be more the rule 
than the exception, I will be looking to become involved in a different 
line of educational inquiry. 

To be more specific about my worries, I would like to divide the 
subsequent remarks into three categories: 1. inquiry and che role of 
information, 2. humanizing data, and 3. accountability vs. responsibility. 
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The first category is really preeminent with the latter two categories 
being corollaries of the position being taken on information systems. 



Inquiry and the Role of Information 

At the outset, it is important to clarify the perspective underlying 
the kind of "information system" that I and my colleagues have envisioned. 
We would not characterize it as a district-driven, "management information 
system" exclusively (or nearly exclusively) oriented around monitoring 
achievement, i.e., norm- and/or criterion-referenced test score results. 
Rather, what we have in mind is more of a school -based, inquiry-driven 
system of information to be collected and used formatively in the process 
of school improvement. 

It is virtually impossible to do justice to the ideas embedded in this 
view of information system in this short report. My hope is that those of 
you interested in these ideas well look at the reports referenced in this 
paper. All I can do here is summarize in shot-gun fashion some core 
concepts: 

1. Outcome indices have limited value, beyond their immediate, 
descriptive signal, for helping direct an agenda for school 
improvement. Certainly I am not suggesting that assessing student 
achievement is unnecessary. Rather, I am suggesting that test 
scores alone are an insufficient basis for action. 

2. What is further necessary is relevant information on the 
circumstances, activities and meanings associated with the 
schooling process. By "circumstances," I mean the whole array of 
structures, situations, and physical features in the school 
(condition of building, teacher-student ratio, student tracking 
policy, materials and resources, and so forth). By "activities," 

I mean the behaviors and processes that constitute the practice of 
schooling (staff decision-making, classroom teaching strategies, 
student socialization patterns and so forth). By "meanings," I 
mean the understandings that people infer from and bring to bear 
upon schooling (orientations like feelings, opinions, attitudes, 
beliefs and values are included here as well as any other 
indicators of meaning—test scores for example). 
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3. The criteria for relevance are based upon the perceived needs of 
the significant "actors" in the setting (e.g., administrators, 
teachers, students, parents) and the inherent value systems 
through which these perceptions are filtered. 

4. Information systems, as knowledge producing and using processes , 
must reflect the variety in: 

a. types of information, e.g., cognitive, affective, contextual. 

b. commonplaces (Goodlad, 1979) of schooling, e.g., physical 
environment, human resources, material resources, organization, 
communication, problem-solving/decision-making, leadership, 
issues/problems, controls/restraints, expectations, climate, 
and curriculum. Curriculum is to be interpreted broadly and 
should include at least these additional commonplaces: 
goals/objectives (academic, personal, social, career), content, 
instructional materials, classroom activities, teaching 
strategies, assessment of student learning, use of time and 
space, and grouping strategies. 

c. domains of information, e.g., individual, class, team/grade/ 
department, school, district, community. 

d. sources of information, e.g., students, teachers, administrators, 
parents, observers, documents. 

e. methods for collecting information, e.g., tests, surveys, 
observations, interviews, case studies, historical analyses, 
document reviews. 

I have tried to pack most of these features into Figure 1 which is offered 
as a heuristic for suggesting the depth and breath of potential school 
information. (Note also that Figure 1 represents just one slice out of 
many possible along the pre-school, K-12, post-secondary continuum.) 

With these concepts as a backdrop, I can now be more explicit regarding 
the theme of this paper: people using or being used by information systems. 
There is a fifth and most important concept to be added to the above list: 

5. Information systems, as knowledge producing and using processes, 
must be multi -paradigmatic, embracing the tensions between these 
alternative inquiry perspectives: 

a. Empirical analytic methods that place a premium on explanation 
via predictive relationships between quantified constructs. 

b. Natural istic/phenomenological methods that place a premium on 
understanding via qualitative interpretations of social settings. 
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c. Critical /dialec tical methods that place a premium on informed 
discourse, values" clarification, and action. 

Thus, a district or school seriously bent upon sustained improvement 
and change efforts will need to involve school staff in the collaborative 
pursuit of understanding— What goes on in their school (s)? How did it come 
to be that way? What are the social, political and economic interests that 
constrain the setting? Reconciling the various phenomenological views of 
the setting and approaching a working consensus on problem areas should 
always the first order of business. As the dialogue proceeds, it will 
become evident that much information is needed— information that can 
be determined through various operational devices (e.g., surveys, tests, 
observations) or information that is already available (e.g., school 
records) but needs to be organized and disseminated. Only when information 
is perceived as useful, should information systems be conceived for use. 

To summarize, information as knowledge is not an end in itself but is, 
instead, a catalyst for evaluative discourse and action; inquiry and 
information need to be legitimized as natural and on-going parts of the 
daily work life of those for whom the knowledge is to be relevant. 

Humanizing Dat a 

Many professional and lay persons both inside and outside of the 
educational research and schooling communities have never been enamored 
with the notion of quantifying the meaning of circumstances and events in 
social settings. To exacerbate matters further, the exponential rise of 
high technology has propelled us into an "age of information." The only 
way to escape being "computerized" is tc disenfranchise oneself from 
economic life. My guess is that these societal changes, coupled with past 
sentiments regarding "research-type" activities, will make those people 
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identified as potential data sources even less sanguine— and mora cynical 

and suspicious— regarding the benefits of the kind of comprehensive information 

systems being described. 

Picking up on the theme from the previous section, a crucial ingredient 
required to overcome this concern is the cultivation of an attitude towards 
information that makes it an intrinsic part of an organizational environment 
that legitimizes pr ofessional inqui>y_by allo^ tjme^ to_ _the 

effort . But there are a few other, more mundane ingredients having to do 
with attitudes toward and techniques of measurement and data analysis . 

Perhaps the most important problem in measurement is to overcome two 
kinds of attitudes that tend to polarize people into either of two belief 
"camps," described by Kaplan (1964) as adherents of either the "mystique of 
quantity" or the "mystique of quality." The extreme position in the former 
camp is embodied in the expression, "If you can't measure it, that ain't 
it." And the opposite extreme in the latter camp— "If you can measure it, 
that ain't it." 

As with all false dichotomies, the truth is somewhere in between and 
is rooted pragmatism. It is unreasonable to believe that the mathematical 
power inherent in numbers somehow transcends the strength (or weakness) of 
their connections with properties they are presumably measuring. It is 
equally unreasonable to assume that numbers assigned to reified concepts 
(such as "self-esteem" and "principal leadership") cannot possibly represent 
anything meaningful. 

The ultimate arbitrator of the meaning of measurement is experience. 
This is why the notions of reliability and validity were invented. To the 
extent that the numbers (i.e., measurements) can be replicated, they are 
reliable. More importantly, to the extent that they serve the measurement 

ERIC ' 287 



6 



9 

ERIC 



purposes intended, they are valid . The key word here is purpose . 
Depending upon the purpose, the evidential arguments for reliability and 
validity may differ. 

Consistent with the primary objective being espoused here for collecting 
and using data— to inform and stimulate staff inquiry— evidence must be 
acquired first for the content validity and second for the credibility of 
information. Content validity is a familiar concept to most people who 
construct achievement tests. It is essentially a rational process of 
matching item content to instructional content, i.e., the course (or unit 
or lesson) objectives and the skills, knowledge, and understandings implied 
by those objectives. Likewise, the items in surveys, interviews and 
observation schedules must be matched to the content they are designed to 
assess. A concept such as "staff cohesiveness" may ba of important concern 
to the organizational work environment in a school. But within the context 
of that school, a concept like "staff cohesiveness" needj to be scrutinized 
for its various meanings. Does it represent support? friendliness? 
trust? respect? morale? commitment? unity? etc.? 

The term "credibility" as a type of validity is used here to epresent 
the degree to which information augments, stimulates, provokes or otherwise 
facilitates meaningful communication between staff in the inquiry process. 
Assessing credibility can only be done by the staff during the course of 
inquiry. Suppose the issue of increasing parent involvement in school 
affairs is under deliberation. Suppose the results of a parent survey 
question regarding the reasons they have for feeling disenfranchised from 
school affairs are added to the information being brought to bear on the 
inquiry. If the results lead to a "so what?" response or a.< inability to 
relate the data to the pertinent issue, it us unlikely that the item will 
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be Included in further surveys. On the other hand, if the results can be 
seen to further the dialogue, the item has proved credible. (See example 
below for Nuvo Elementary School). The point here is that issues pertaining 
to the validity of information are not exclusively those facing psychometri- 
cians. These issues must be continually addressed as people attempt to 
make sense out of the information collected. 

This leads, then, to the kinds of data analyses likely to be of use to 
staff for facilitating their inquiry about the conditions of their schools 
and the possible avenues for improvement and change. It is unlikely that 
the results of multivariate analyses such as multiple regression, covariance 
structure modelling, and the like, will be of use in this effort. Obviously, 
such analyses can be useful for research purposes in studying complex 
relationships between variables and how they change over time. Yet, they 
lack the immediacy and simplicity required to inform staff deliberation on 
specific problems and issues. 

But do not confuse simplicity with simple-mindedness. The myth that 

complex numerical manipulations somehow yield better, more "scientific" 

results has no place in the rationale for a "people-oriented" information 

system. The pow r of a single percentage should not be underestimated as a 

stimulus for facilitating and advancing the dialogue. A brief summary of 

the events of a staff meeting at one elementary school should illustrate 

how simple tabulations of data can facilitate staff inquiry. 

A continuing issue at Nuvo Elementary School concerned curriculum 
balance and the role of content area specialists. Prior to this 
meeting it had been suggested that staff really didn't know how much 
time was being devoted to various subject areas in each grade level. 
As an approximation to this bit of missing knowledge, staff responded 
to a question asking for the approximate, weekly number of hours 
allocated to a number of subject area divisions. Since teachers at 
this school taught in 10 teams (of 2-3 teachers each) spread across 
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grade levels, teams (rather then individuals) reached consensus on * 
this item; and the 10 team responses were arrayed and presented as 
input to the staff meeting. 

>» 

Preliminary discussion began around the nature of the item itself and 
the difficulty of cutting up the hours of the day to correspond to the 
subject matter categories. Thus, to some extent, the hours indicated 
by teams were not realistic. Yet, all tsams felt that the general 
patterns in the data "rang true." These patterns were two-fold: 
(1) There were extreme imbalances in the time allocated to different 
content areas and (2) The nature of those imbalances were very different 
in different grade levels and teams. These observations fed back 
nicely into the major thrusts of the issue. First, what ought be the 
curricular balance between subject contents, should it be different at 
different grade levels, and, if so, how can balance be maintained in 
the continuum from one grade level to the next? 

But the original criticism of the survey question really highlighted a 
second thrust. How separable are content areas, and to what extent do 
we (and should we) teach subjects (e.g., reading, math and science) 
together as they naturally occur within a thematic unit (e.g., ecology)? 
This query, of course, raised the role of content specialists as being 
"outside class" resources versus being regular members of team with 
special talents that can be shared with other staff as needed. 

This is enough of a scenario to make the point regarding how simple v 

(not simplistic) survey results can facilitate inquiry. It should also be ' 

noted that content validity and credibility issues were implicit in this ^ 

scenario and could be made explicit during the course of the inquiry. 



Responsibility vs. Accountability 

Teachers, students, parents, etc. have been "burned" far too often by 
mindless exercises of data collection (usually surveys), the results of 
which never see the light of day or, if they do, are presented in a useless 
form, in a useless setting, and/cr at a useless time. Even worse, however, 
are the scars left from using information as a weapon, often under the 
rubric of accountability . Most often, the arsenal is composed of test 
score information, but I have verified "war stories" where other types of 
information (e.g., teacher educational values/beliefs; instructional 
practices; etc.) have been used against people (e.g., RIF decisions). 
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I find this attitude towards and use of information to be intolerable* 
It is not surprising, then, that it is wholly incompatible with the view 
being espoused here of information systems as an adjunct to and by-product 
of a school -focused inquiry and change process. From this perspective, 
"evaluation is no longer a tagged-on prerequisite; it is intrinsic to the 
reflective process and is engaged in by all involved. "Accountability" is 
recast to a higher ethical level; people become accountable to ne another 
rather than to what are often seen as arbitrary, outside interests. 
Responsibility would seem to be a more reasonable term to characterize this 
view of inquiry and the use of information. 

Summary 

There are at least three issues that must be taken seriously by 
information system advocates if the systems are to be used by and for 
people instead of against. them: 

1. Intimately involving people in developing the purposes, contents 
and uses of the systems, 

2. Developing timely, relevant and straightforward techniques for 
bringing information to bear upon practice, 

3. Creating a climate of mutual trust and responsibility for 
generating and using information. 
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Footnote 



The following reports provide a more thorough discussion of all 
concepts embedded in this paper: Sirotnik, 1984; Sirotnik and 
Burstein, 1983; Sirotnik and Oakes, 1981 and 1983, 
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